Cutting Agent Costs Without Cutting the Product

When an agent bill comes in higher than forecast, the first instinct is to shop for a cheaper model. That is almost never where the money went. The bill is the sum of every model turn, every tool call, every retry, every search, and every workflow a user abandoned halfway through — and the advertised per-token price of one model explains almost none of it.

Agent cost is an architecture and product-behavior problem. Treat it as a procurement problem and you will negotiate a discount on a system that is still wasting most of what it spends.

Measure cost per completed outcome

Attach every unit of usage to a task identifier, and record whether that task completed, needed human correction, or was abandoned. Then divide. A workflow that looks inexpensive per turn can be the most expensive thing you run if it loops, retries, or escalates on a third of attempts.

This single change reorders most teams’ priority list within a week, because it makes visible the difference between work that finished and work that merely happened.

• Route routine steps to the cheapest model that still passes its eval
• Cache stable instructions, retrieved context, and deterministic tool results
• Summarize large tool outputs before they re-enter the loop and get paid for again
• Set turn, time, and spend budgets with a graceful stop rather than a hard cut
• Cancel work when the user leaves or the outcome is no longer achievable

Stop paying to rediscover the same facts

Watch a trace and you will find the agent working out the same rules on every run — which fields are required, what the valid status values are, what order operations must happen in. You are paying reasoning tokens to reconstruct application logic that has not changed since it was written.

Push that knowledge into tool schemas, workflow state, and deterministic validation where it belongs. The model should spend its effort on the part of the problem that is actually variable — the customer’s specific situation — not on rebuilding your business rules from scratch forty times a day.

The retry loop is where budgets die

A retry costs more than the original attempt, because the failed context usually comes along for the ride. Three silent retries with growing context can cost several times a first-attempt success. Cap retries explicitly, change strategy between attempts instead of repeating the same call, and escalate to a stronger model or a human rather than looping — an agent quietly trying the same thing five times is the most expensive failure mode there is.

Do not let the budget degrade the product silently

The worst way to enforce a limit is invisibly: truncated context, a quietly downgraded model, an answer that is subtly worse for reasons the user cannot see. They will not know the system was constrained. They will just conclude it is not very good.

“When a limit is reached, keep the completed work, say plainly what remains, and offer a clear path to continue at the right level of intelligence.”