Production Agents Run on Policy, Evals, and a Clean Handoff

Any decent model can hold a convincing support conversation in a demo. The demo is not the hard part. Production starts the moment the system has to follow a company policy it did not write, touch only the records it is cleared for, take an action that costs money, notice when it is out of its depth, and pull in a human before a mistake becomes a refund or a lawsuit.

One job, one owner, one testable contract

Write down four things before any code: the request this agent handles, the knowledge it may read, the actions it may take, and the name of the person accountable when it gets one wrong. A narrow job is a contract you can test against. “Helps customers with anything” is not a scope — it is an evaluation problem with no boundary, and it will fail in a direction nobody predicted.

Separate the policy from the personality

Teams routinely mash operating rules and conversational tone into one enormous prompt, then wonder why a copy tweak changed a refund decision. Keep them apart. Policy is data — versioned, reviewable by the people who own the process, testable independently. Tone is presentation. When the two live in the same string, every wording change is an untested policy change.

• Represent operating policy as structured, versioned data — separate from voice and style
• Grant only the tools and data the defined job requires, and nothing “just in case”
• Require approval for high-impact, unusual, or irreversible actions
• Escalate with the transcript, gathered context, and attempted steps attached
• Feed every production failure back into a permanent evaluation set

Grade the decision, not the prose

This is where most eval suites are useless. They score fluency, and fluency is the one thing modern models are guaranteed to deliver. A perfectly written answer can cite the wrong policy, call the wrong tool with valid-looking arguments, or stall past the point where a human should have taken over.

Score the operational facts instead. Did it retrieve the correct source? Did it stay inside the permission boundary? Did the approved action actually complete? Did it hand off when it lacked authority — and did it do so before the customer got frustrated, not after? Language quality is a product concern. Operational correctness is what decides whether you can scale this thing.

The escalation nobody designs

Most teams build escalation as an afterthought: a transfer that drops the customer into a queue with none of the history. That is worse than no agent at all, because the customer already spent three minutes explaining. Treat the handoff as a product surface. The human should open a view with the transcript, the intent the agent inferred, what it already checked, what it changed, and what it was uncertain about.

Ship changes the way you ship code

Real sessions and real escalations are the best source of improvements you will ever have. Use them to propose specific changes to policy, prompts, retrieval, or tool schemas. Then test each change against the same representative cases, diff it against the version currently in production, and roll it out on purpose.

“Dependable agents get better every week. The business stays in control the entire time. Those two things are not in tension — they are the same discipline.”