Circuit Breakers: Making Agent Tool Calls Safe to Skip

The payments vendor's status page is amber, but the agent doesn't read status pages. Its refund_charge call times out at the 30-second deadline. The retry policy fires — correctly — and times out again. The model, told "tool failed," does the reasonable thing: it rephrases the arguments, splits the refund, tries get_charge first to "verify the charge exists." That call times out too. Nine tool calls later the run has burned four minutes of wall clock, several thousand tokens of failure text, and a context window now half full of stack traces. The model starts answering around the gap — summarizing a refund it never executed — and the session ends with the operator reading a transcript that looks like the tool worked.

Nothing crashed. Every component followed its design. The retries were right, the rephrasing was right, the timeout was right. What was missing was memory: the second timeout against that vendor carried no information from the first, so the harness spent a full retry-and-rephrase budget discovering, call by call, what one counter could have said upfront — the dependency is down, stop paying for it.

This is the circuit-breaker chapter of the tool-reliability series. Retries got idempotency keys, ordering got write serialization, deadlines got timeouts, load got backpressure. This one is about the caller's right to stop trying: how an agent harness decides a tool is not worth calling, what the model deserves to be told while that verdict stands, and how that verdict survives across sessions.

What a down tool costs an agent, not a request

A plain HTTP client hitting a dead dependency wastes a connection and some latency. An agent wastes considerably more, because the failure loops back through the model:

• Latency compounds through the turn. Each failed call consumes its full timeout, then a backoff, then a retry timeout. Three attempts against a 30-second deadline is 90-plus seconds the user watches a spinner — for an answer the breaker could have returned in microseconds.
• Failure text pollutes the context. Every timed-out call leaves a tool result in the transcript, and stack traces and retry notices are the worst token-to-value ratio in the window. By the fourth failure the model is reading more about the tool's failure than about the user's problem.
• The model compensates creatively. Tell a model "error" enough times and it improvises: alternative arguments, alternative tools, alternative orderings. Some of that is exactly what you want. Some of it is a refund attempted through a path nobody reviewed. A model routing around a known-dead tool is a design feature; a model probing a dead tool because nothing told it the tool is dead is an outage amplifier.
• The discovery repeats. Agents increasingly run as long-lived services, not one-shot scripts. If session 47 learned the vendor was down at 09:12, session 48 at 09:40 should not relearn it through its own fifty calls.

A circuit breaker puts a counter where that memory belongs: after enough real failures, further calls fail fast at the harness layer, never reaching the network and never reaching the model as anything but a structured, actionable result.

The breaker belongs to the harness, not the model

The breaker wraps tool execution. It is not a prompt instruction ("if the payments tool keeps failing, stop using it") — instructions are advisory, cost tokens, and degrade exactly when the context is most polluted. The breaker is executor-level state that the model cannot bypass and never has to remember.

class BreakerOpen(Exception): ...

def call_tool(tool, args, credential):
    key = (tool.name, credential.id)
    br = registry.for_key(key)
    if br.is_open():
        # fast-fail BEFORE the retry policy: an open breaker is not
        # a transient failure, and retrying it just multiplies the
        # latency the breaker exists to prevent.
        raise BreakerOpen(retry_after_ms=br.ms_until_probe())
    try:
        result = execute_with_retries(tool, args, credential)
    except DependencyPain as e:      # see next section
        if br.record_failure():
            notify(f"breaker open: {key} after {br.failures} failures")
        raise
    br.record_success()
    return result

Two scope decisions matter more than the state machine itself:

• One breaker per (tool, credential) — or per (tool, backend host) when a tool fronts one vendor. Two tools that both call Stripe should share a breaker, because they share the failure domain. One tool parameterized over two workspaces with separate credentials should not — tenant A's outage is not tenant B's. Scope the breaker to the dependency, not to the name in your tool registry.
• The breaker sits inside the retry policy, not around it. A single call may retry transient blips freely; the breaker counts exhausted retry budgets (or individual failures — pick one and pin it in a test), so one flaky exchange never opens a circuit. Trip on patterns, not on events.

Open on dependency pain only

The classic mistake is counting every exception. The breaker exists to protect a dependency and the caller from each other when the dependency is unhealthy. Classification, in agent terms:

Trips the breaker: connection refused and resets, TLS and DNS failures, timeouts (the dependency's clock ran out), 5xx, and — the agent-specific entry — malformed tool output. A tool that returns a 200 with a body your schema validator rejects is a failing dependency exactly as much as a 503 is; the model cannot use the response, and hammering it won't help — though it is usually a deploy-shaped failure, so the half-open probe after the vendor's rollout finishes will pass where a retry storm would have paid full price all morning.

Does not trip: 4xx validation errors, schema rejections of the arguments, and permission denials. These are caller bugs — the model sent wrong arguments, or the credential lacks a scope. Tripping on them hides defects behind "tool unavailable" and the schema bug you needed to see ships quietly. A 400 should surface to the model as a fixable argument error, not disappear into an open circuit.

Feeds the cooldown instead of the counter: 429 with Retry-After. Rate limiting is the server telling you the schedule, not reporting its health — the backpressure chapter's domain. But a 429 whose retry-after exceeds the call's own deadline is a failure in the agent's terms, and three of those in a row should open the circuit with the cooldown set from the last Retry-After seen.

Auth failures deserve their own sentence: a 401 on a credential that worked an hour ago means the credential was revoked — that's an alert-and-pause event, not a breaker state, and batching it into the failure counter both hides the real cause and delays the probe that would never succeed anyway.

What the model sees when the circuit is open

The model must never see a bare "tool failed" for an open circuit — that's the input for another rephrasing storm. It sees a result shaped like the rate_limited result from the backpressure chapter, because the model's correct move is the same: route around it.

{
  "status": "circuit_open",
  "tool": "refund_charge",
  "reason": "payment vendor unreachable (5 consecutive failures, last: timeout after 30s)",
  "retry_after_ms": 120000,
  "alternatives": ["issue_refund_manually"]
}

The fields earn their tokens. retry_after_ms lets the model decide whether to wait, and a model told "two minutes" will often tell the user "two minutes" instead of spiraling. reason in plain words prevents the model from inventing a cause. alternatives, when a genuine fallback tool exists, turns a dead end into a route. What the model does with this — degrade gracefully, answer partially with the gap stated, or hand off — is a product decision you've now made possible rather than impossible.

Half-open, with a fan-out at the door

After the cooldown, the breaker admits one probe to test the dependency. Agents make this interesting: the model, told circuit_open twice, may come back next turn with five parallel calls to the same tool. Half-open must admit exactly one probe and fast-fail the other four with probe_in_flight — a distinct status, so the model knows the question is being answered and doesn't conclude the tool is still dead from four microsecond rejections.

A failed probe re-opens with an exponentially growing cooldown, capped (say 30s → 1m → 5m → 15m): a vendor having a bad morning shouldn't get probed every thirty seconds all day. A successful probe closes the circuit and the counters reset — and if your executor fans out, let the next calls through immediately rather than staging them; the probe just proved the path.

Breaker state that outlives the session

For agents-as-services, in-memory breakers reset on every deploy and restart, which is precisely when dependencies are likeliest to be flaky. Persist the minimal verdict — per key: state, failure count, opened_at, last_retry_after — to a small state file or store the harness owns, with a staleness bound: persisted state older than the cap cooldown is discarded (a verdict from an hour ago says nothing now). On startup, a fresh session inherits "vendor down, probe at 12:04" instead of relearning it the expensive way.

Keep two caveats in view. Persistence is advisory inside the process — the in-memory breaker stays authoritative, the store just seeds it. And with N harness replicas, each holds its own breaker, so a recovering dependency sees N probes at cooldown expiry, not one. That is acceptable probing pressure by construction, and trying to elect a single prober across replicas buys you a distributed-coordination problem worth more than it saves.

It composes with the rest of the series

The breaker is the last piece because it consumes the others' signals. Timeouts define the failure clock that feeds the counter; idempotency keys make probes safe to issue (a probe that times out after executing must not double-run on its own retry); Retry-After from the rate-limit layer sets the initial cooldown; the command ledger records breaker_open and breaker_closed as first-class events with their counts; and the observability chapter's rule applies with full force — a breaker transition is one of the highest-signal events your telemetry will ever carry. An open that persists two cooldown cycles on a production tool should page a human, because the breaker is reporting the outage, not causing it.

A breaker you can test

Like every chapter in this series, the proof is a test that counts requests rather than outcomes:

def test_breaker_opens_and_skips_the_network(fake_payments):
    fake_payments.fail_mode = "timeout"
    for _ in range(4):                      # retry budget x calls
        with pytest.raises(ToolError):
            harness.call("refund_charge", {...})
    assert fake_payments.request_count == 6  # the real attempts, counted
    result = harness.call("refund_charge", {...})
    assert result["status"] == "circuit_open"
    assert fake_payments.request_count == 6  # nothing new hit the wire

def test_validation_errors_never_trip(fake_payments):
    fake_payments.fail_mode = "400_invalid_args"
    for _ in range(10):
        harness.call("refund_charge", {"bad": "args"})
    assert fake_payments.request_count == 10  # still reaching the tool

Inject the clock for cooldown tests (a real-sleep half-open test is a flake factory), assert the fan-out-under-half-open case admits exactly one request, and one test that boots a fresh harness against persisted state and asserts the first call fast-fails with circuit_open. Then the negative test above — the one that proves argument errors can't open a circuit — is the difference between a breaker that protects the dependency and one that buries your bugs.

What this does not fix

A breaker does not heal dependencies; it buys honesty about their absence. It doesn't replace retries for blips (it sits above them), it doesn't decide what the agent should do without the tool (it makes that decision survivable), and it does nothing for one-shot scripts with no future calls to protect. If every tool call in your product is a single lonely request, skip the breaker; the moment agents run in loops and sessions, it's the cheapest reliability you'll ever add.

The checklist

• Breaker in the harness, wrapped around retry budgets, scoped to (tool, credential) or the shared backend — never to the tool name alone.
• Trips on dependency pain only: transport, timeouts, 5xx, malformed output. Argument errors and permission denials stay visible to the model.
• 429 with Retry-After sets the cooldown; consecutive deadline-exceeding 429s open the circuit.
• Open means a structured circuit_open result with retry_after_ms and plain-word reason — never a generic error.
• Half-open admits exactly one probe; parallel callers get probe_in_flight. Failed probes grow the cooldown, capped.
• Verdicts persist across sessions with a staleness bound; fresh runs inherit outages instead of rediscovering them.
• Transitions are ledger events and telemetry; an open past two cooldowns pages a human.
• CI counts requests: the wire is the witness. And the 400s-can't-trip test ships with the breaker.

An agent that keeps calling a dead tool isn't persistent — it's uneconomical, and eventually creative in ways nobody asked for. The circuit breaker is the small piece of memory that turns "keep trying" into "try again when there's reason to believe," and it costs less than one timeout to own.