Why Most AI Agents Break When You Rely on Them

A lot of businesses see an AI demo, get impressed, and assume it's ready to run part of their operations. In practice, there's a big gap between an AI that can hold a conversation and an AI system you can actually trust to handle real work, unattended, every single day. We recently watched a LangChain engineering talk that lays this gap out clearly, and it's worth translating into plain terms for anyone thinking about bringing AI into their business.

An AI agent usually means less than you'd think

Most people picture an AI agent as a chatbot that can also take actions: send an email, update a spreadsheet, look something up. That's true, but it's only the visible part. The AI model itself, the part that actually "thinks," is really only half the story. Everything built around that model, the rules, the memory, the error handling, the guardrails, is what decides whether the agent is actually dependable or just impressive in a demo.

The talk put it simply:

If you're not the model, you're the harness.
LangChain, "Building Deep Agents and Deploying in Production"

That surrounding system is called a "harness," and it's a bigger deal than the name suggests. As one example from the talk: the tooling behind Claude Code, a well-known AI coding tool, runs to roughly 500,000 lines of code. Almost none of that is the AI model itself. It's all the scaffolding that makes the model reliable enough to actually use.

Concretely, that scaffolding is things like: retry logic for when a step fails partway through, rate limiting so a runaway loop can't rack up an unbounded bill or spam an external system, validation that checks the model's output is actually structured the way the next step expects, sandboxing so an action can't touch more than it's supposed to, and logging detailed enough that someone can reconstruct exactly what the agent did after the fact. None of that is glamorous. All of it is the difference between a demo and something you'd trust with real customer data.

That ratio, a small model wrapped in a large amount of scaffolding, is roughly what to expect from any agent built seriously. If a vendor's entire pitch is "we connected model X to your CRM," with nothing said about failure handling, permissions, or logging, that's usually a sign the harness hasn't been built yet, and you're being sold the 5% that's easy to demo.

The real risk: an agent that works in testing but fails on real work

Here's the scenario that matters for a business. Say you have an AI agent working through a genuinely multi-step task: following up with a batch of leads, processing a stack of invoices, or qualifying enquiries as they come in. That's not one action, it's dozens of small steps in sequence.

Now say something goes wrong on step 40 of 60. A cheaply built agent has no memory of what it already did. It either stops entirely, or worse, starts over from step one, potentially double-emailing leads or re-processing invoices it already handled. A properly built agent picks up exactly where it left off. That difference sounds small until it's your customer data or your invoicing on the line.

There's a quieter version of this failure that's arguably worse: an agent that doesn't stop or restart, but silently retries the same failing step in a loop. Nothing looks broken from the outside. The task just never finishes, and depending on what that step touches, it might be quietly running up an API bill, hammering an external system, or sending the same message repeatedly, until someone happens to notice, usually from a complaint or an invoice, not a warning from the agent itself.

What to actually check before you trust an AI agent with real work

Based on what the talk covers, here's a plain-language checklist worth applying to any AI agent, whether you're building one or being sold one:

Can it recover from a failure partway through, without restarting the whole task or repeating actions it already took?
Does it have clear limits on what it's allowed to do on your behalf? An agent sending one email for you is fine. An agent that could silently send a thousand without anyone noticing is a different problem entirely.
Does it keep a human in the loop for anything that actually matters, rather than running quietly in the background and telling you after the fact?
Does it remember relevant context over time, like your preferences or past conversations, instead of starting from zero every session?

Notice that none of these four questions are really about the model. They're all about the boundaries around it: how it recovers, what it's allowed to touch, when it stops for a human, and what it remembers. That's a useful shortcut when you're evaluating a pitch, ask about the boundaries first, since that's what most sales conversations skip straight past on the way to the demo.

What "built properly" actually costs

It's worth being clear that "built properly" doesn't mean the most expensive model or the most infrastructure. Often the opposite is true: a narrowly scoped agent running on a modest, cheaper model, with clear guardrails, solid error handling, and a human checkpoint where it actually matters, will outperform an expensive model wrapped carelessly around a task it was never scoped for. The spend that actually buys reliability goes into the harness, not into upgrading the model for its own sake.

This is also why scoping matters more than model choice. An agent asked to handle "customer service" end to end, with no defined boundaries, is set up to fail somewhere eventually, simply because the surface area is too large to guard properly. An agent asked to handle one specific thing, tag incoming leads by urgency, draft (not send) a reply for review, flag anything it isn't confident about, is small enough that every one of the four checklist questions above can actually be answered with a yes, not a shrug.

A pattern worth recognizing: the demo-to-production gap

It's worth naming why this gap exists in the first place, since it explains a lot of disappointing AI rollouts. A demo only has to work once, in front of an audience, on inputs the presenter already knows will behave well. Production has to work correctly on the hundredth run, the thousandth, including the malformed input, the API that times out, the edge case nobody thought to test. Almost every part of the harness described above exists specifically to close that gap, and almost none of it is visible in a five-minute demo, which is exactly why it's so easy to sell around.

Why this matters if you're considering AI for your business

None of this means AI agents aren't ready for SMEs. The opposite, actually: the underlying technology has gotten reliable enough that, built properly, an AI agent can genuinely take repetitive work off your team's plate. But "built properly" is doing a lot of work in that sentence. If someone pitches you an AI agent for your business, the four questions above are a reasonable way to tell a serious build from a demo that hasn't been stress-tested yet, and the honest answer to some of them, at least at first, might simply be "not yet, here's what that would take."

Watch the full talk: "Building Deep Agents and Deploying in Production" (LangChain) ↗