You Will Find Out About The Runaway Agent From The Invoice

A research agent gets a web_search tool. The tool does its job well: every result mentions three more competitors worth investigating. The model, being thorough, pulls the thread.

Twenty turns later it is still going. Each turn re-sends a context that grew since the last one, so cost climbs quadratically while the run looks perfectly healthy — no errors, no timeouts, green dashboard.

You find out from the invoice.

That is the part worth sitting with. Detection is not a control. By the time a token counter shows the overspend, you have already paid for it.

TL;DR

Three things most cost implementations get wrong:

1. Check before the spend, not after. The budget hook halts from before_model, because refusing a tool cannot stop an expensive model call that has not happened yet. In the demo the halt fires with model.calls == 2 — the third call was never made.
2. Reserve, then reconcile. You do not know a completion’s cost until it returns, so charge a pessimistic estimate up front and settle to actual afterwards. A budget that only counts real usage always overshoots by exactly one call, and that call is the expensive one.
3. Per-run budgets alone are useless. A $0.05 cap per run does nothing about 10,000 runs. You need a second window keyed on the tenant, and the tenant window is the one that saves you.

Plus one product decision: exhaustion should degrade, not crash. “Here is what I found before I ran out” is a product. A stack trace is an incident.

OWASP: ASI08 (Cascading Failures) — the economic variant. Runnable code: patterns/harness/cost_tool_budgeting/

The runaway, bounded

uv run python -m patterns.harness.cost_tool_budgeting.demo

=== WITHOUT the pattern ===
  status: completed, tool calls: 20
  nothing stopped it; cost is whatever the loop decided

=== WITH the pattern ===
  status: halted
  reason: model calls budget exhausted (13/12) — stopped before the call, not after
  executed tool calls: 7 (budget 8)
  spend: $0.0346 (budget $0.0500)
  output: So far: 12 competitors found, pricing between $19 and $99/seat.
          (Stopped early: run budget exhausted. This answer may be incomplete.)

  the agent was warned before being stopped:
    denial -> wind-down

=== the per-run budget that isn't a budget ===
  every run stayed under its own $0.50 cap, but run #2 hit the
  tenant cap of $0.15 — cumulative spend $0.1416

That last block is the argument. Every individual run was well-behaved. The aggregate was not.

The pattern

guard = BudgetGuard(
    run_budget=Budget(model_calls=12, tool_calls=8, usd=0.05),
    tenant_budget=Budget(usd=50.0),      # shared across every run for this tenant
    wind_down_at=0.8,                    # deny tools at 80%, halt at 100%
)

Wind down before hard-stopping. At 80% the guard denies tools with “answer with what you already have,” so the agent gets a turn to summarise. At 100% it halts. The difference between those two behaviours is the difference between a degraded answer and no answer, and it costs about six lines.

A halted run still returns its best partial output. The guard walks back for the most recent assistant prose and labels it incomplete.

Zero means unlimited, so adopting this does not require pricing every axis on day one.

When to use it

• Any agent with an unbounded loop — which is any agent whose stopping condition is the model’s judgment rather than a fixed pipeline.
• Any multi-tenant product. Cost per tenant is a product metric and an abuse signal before it is a finance problem.
• Any agent with tools whose output can grow its own next input: search, crawling, file listing, log queries, other agents.
• Before your first public launch, not after your first surprising invoice.

When NOT to use it

• Fixed-step workflows where the number of model calls is a constant. Cost is knowable at design time; a budget object adds a failure mode and tells you nothing.
• As your only cost control. Provider-side spend caps, per-key rate limits, and alerting all still apply. This bounds one run’s behaviour, not a compromised key or a bad deploy.
• Do not set budgets from vibes. A cap tight enough to break legitimate long tasks trains your team to raise it reflexively, which is how you end up with usd=1000 and no control at all. Measure the p95 of real runs first, then set the cap above it.
• Do not confuse this with rate limiting. A rate limit protects the provider from you and belongs at the gateway. A budget protects you from your own agent and belongs in the harness. Neither substitutes for the other.

Trade-offs and failure modes

• The token estimate is wrong on purpose. Four-chars-per-token, rounded up: cheap, deterministic, and biased toward over-charging. Swap in the provider’s tokeniser if you need accuracy, but keep the rounding direction — an optimistic estimator is an overspend generator.
• Prices belong in config, not code. Rates change more often than deploys, and cached reservations computed from stale prices silently under-charge.
• The in-memory ledger is a teaching device. A per-tenant budget must be shared across every process serving that tenant, which means Redis or a metering service, which means atomicity: two concurrent runs can both pass the check and both charge. Use atomic increments and accept a small overshoot, or serialise and accept latency.
• Reservations leak on crashes. If a process dies between charge and reconcile, the reservation stays on the ledger and the tenant is over-charged until something expires it. Give reservations a TTL.
• Wind-down changes agent behaviour in ways your evals never see. A run that hit 80% produces a worse answer than the same run at 20%, and offline evals never reach the threshold. Log wind-down rate as a quality metric, not just a cost one.
• Budgets and retries interact badly. A flaky tool can exhaust a run’s entire allowance on one failing call. Compose with failure containment, which stops the retry from happening at all.

Frequently asked questions

Should the budget be in tokens or dollars?

Both, and also in calls. Tokens are what the provider meters, dollars are what finance asks about, and call counts are the cheapest way to catch a loop. They fail in different situations — a loop of tiny calls blows the call budget long before the token budget.

Where do I get the numbers to set a budget?

From production, not from estimation. Ship with generous caps and full metering, look at the distribution after a week, then tighten to something above p95. Setting caps before you have data produces either a cap that never fires or one that fires constantly.

What about caching — does that change the maths?

Yes, significantly, and in your favour: prompt caching makes the growing-context problem much cheaper. It also interacts with context compaction, because rewriting the front of the context invalidates cached prefixes. Compact from the middle and keep the head byte-stable.

Is a hard halt too aggressive for user-facing agents?

That is why wind-down exists. The halt is the backstop; the wind-down is the behaviour users actually experience. If you find yourself wanting to remove the halt, raise the cap instead — an unbounded loop with no terminal condition is how you get a five-figure surprise.

References

• OWASP GenAI Security Project, Top 10 for Agentic Applications 2026 — ASI08
• Anthropic, Building Effective Agents — the economics of agentic loops
• Microsoft, Agent Governance Toolkit — “Agent SRE”: error budgets and SLOs for agent runtimes
• Runnable code and tests: patterns/harness/cost_tool_budgeting/

Part of the agent harness and governance series. Next: failure containment — the retry loop a circuit breaker cannot see.