A demo has to answer one question: does this work when everything goes right. Production has to answer three more: who approved this action, what happens when it fails, and can you prove what the agent did three weeks after the fact. Most AI pilots die in the gap between those two lists, not because the model got worse, but because nobody built the second list until the demo was already scheduled in front of the executive team.
The pattern is consistent enough to predict. A vendor or internal team wires an agent to a live system, feeds it a clean scripted scenario, and it performs well. Everyone in the room agrees it's impressive. Then someone asks what happens if the agent gets it wrong on a real customer's account, and the room goes quiet. That question is where most pilots stall out permanently.
The demo never has to say no to itself
A demo runs on a golden path. The input is clean, the API calls succeed, the data is exactly what the agent expects. Nobody demos the ticket with malformed fields, the API that times out mid-task, or the edge case where the "obvious" action is actually the wrong one. That's not dishonest — it's just what a demo is for. But it means the demo has never once had to decide what happens when it shouldn't proceed.
Production is the opposite condition. Real data is messy, real systems fail intermittently, and real actions have consequences that can't be undone by refreshing the page. An agent that issues refunds, edits customer records, or sends emails needs an answer for "what happens when this specific action is wrong" before it runs once unattended, not after. Most pilots don't have that answer because nothing in the demo ever forced the question.
The result is a familiar failure mode: the pilot works in the sandbox, gets scheduled for a "small" production rollout, and then stalls the first time it touches something that matters — a real invoice, a real customer email, a real account change — because nobody who built the demo also built the parts that only matter when something goes wrong.
The record has to exist before the question does
Getting an agent to production reliably means treating approval, audit, and failure handling as part of the build, not as hardening you add after the pilot succeeds. That maps to two of the five things every agent deployment needs to get right: governance and audit.
Governance means the agent's authority is defined as policy before it acts, not decided in the moment by whoever is watching the dashboard. Some actions — drafting a response, flagging a record, summarizing a document — can run without a human in the loop. Others — issuing a refund, changing a permission, sending an external email — should stop and wait for a specific person to approve, every time, regardless of how confident the agent is. That line has to be drawn deliberately, tied to what the action actually does, not to how the demo happened to be scripted.
Audit means every action the agent takes is written down with what it did, what policy allowed it, and why — at the moment it happens, not reconstructed afterward from logs that were built for debugging, not for accountability. A stack trace tells you an API call failed. It doesn't tell you why the agent decided to make that call, what alternative it considered, or who would have been asked to approve it if the policy had required a human. When a customer or a regulator asks why an agent did something specific, "we'd have to check the logs and get back to you" is the answer that ends the pilot.
This is also where identity earns its place on the list. An agent acting under a shared service account can't be the subject of an audit record that means anything — you can't attribute an action to "the agent" if the same credential is also used by three other processes and two humans. A scoped identity per agent is what makes the audit trail actually traceable to a specific decision-maker, human or otherwise.
At Blackwall Labs, this is the difference we build for from the start of an engagement: every agent gets a scoped identity, every consequential action runs against an explicit approval policy, and every action is written to a record at the moment it happens — inside the customer's own infrastructure, not a system we host. It's not a layer added once a pilot is declared ready to scale. It's the thing that determines whether it's ready.
Where this gets genuinely hard
None of this is free, and pretending otherwise is its own failure mode. Approval gates set too loosely don't actually protect anything — if every action auto-approves after five seconds of silence, you've built a audit trail for a rubber stamp. Set them too tightly and the agent stops being useful; a human reviewing every single output is just the old manual process with extra latency.
Finding the right line takes real judgment about the specific action, the specific business, and the actual cost of getting it wrong — and it changes as trust in the agent's track record builds. Multi-agent orchestration has a similar trap: coordinating several agents can remove real toil, but it can also just relocate the risk into the handoffs between them, where failures are harder to trace than in a single do-everything system. There's no default policy that gets this right for every company; it has to be built against how the business actually operates, not against a template.
The question to sit with
If an agent your team is piloting right now took an action on a real customer's data today, and someone asked next month exactly why it did that — not what the log line says, but why, under what approval, against what policy — could you answer in the time it took them to ask? If the honest answer is "not yet," that's not a reason to stop the pilot. It's the actual definition of what's left before it's ready. Book a diagnostic call if you want a second set of eyes on where the gap is.