Tool Call Timeouts and Cancellation: Making Agent Tool Calls Safe to Abandon

An agent kicks off export_customers with last quarter's filters. The tool usually returns in four seconds. Tonight the warehouse is behind, and at thirty seconds the harness deadline fires: the HTTP connection is torn down, the model is handed a generic "tool failed" message, and the transcript moves on. The export keeps running on the server. Eleven minutes later it finishes, writes a 2 GB file to shared storage, and bills the account for the full scan. Nobody tells the model. Two steps later the plan needs the export, can't find it, and starts a second one.

Nothing in that story is a hallucination or a bad model decision. Every individual component behaved as designed — the harness bounded its wait, the tool did its job, the model reacted to what it was told. The bug is in the contract between them: a timeout was reported as a failure, a cancellation was reported as nothing, and an abandoned operation was left to haunt the system as unrecorded work. Retries got the idempotency-key treatment in an earlier piece; ordering got write serialization; result shapes got response design. This article is the time-domain sibling: how long a tool call may wait, what happens when it stops, and what the model deserves to know about it.

A timeout is a client-side opinion

When a harness times out a tool call, exactly one thing happens: the client stops waiting. Everything else is unknown. The server-side truth after a client timeout is one of three states, and they demand completely different reactions:

• The request never arrived. The connection stalled before delivery; no work ran. A retry is free.
• The work completed, and the reply was lost. The operation happened; only the response is missing. This is the most common shape for timeout failures — the failure sits on the response path, not the execution path.
• The work was interrupted mid-flight. Partial state: half-written batches, a held lock, a job marked running forever.

A generic {"error": "tool timeout"} erases that distinction. The model, told the tool failed, will rationally do the worst available thing for state two: start a fresh export with a fresh identity, doubling the cost and the artifact pile. The fix is not a smarter model — no amount of reasoning recovers information the harness deleted before rendering. The fix is a contract that preserves what actually happened.

The deadline budget

Most harnesses configure one number: TOOL_TIMEOUT_SECONDS = 30, applied to every call. That default fails in both directions. Read-only lookups get thirty seconds to hang; a run that has already burned nine minutes of its user's patience gets a fresh thirty-second slice per tool for as long as it keeps calling. Timeouts should be derived from a budget, not minted per call:

class RunBudget:
    """One wall-clock budget for the whole run; every call spends from it."""

    def __init__(self, total_ms=600_000):
        self.deadline_ms = monotonic_ms() + total_ms

    def remaining_ms(self) -> int:
        return max(0, self.deadline_ms - monotonic_ms())


def execute_tool(tool_call, budget: RunBudget, registry):
    spec = registry.resolve(tool_call.name)
    deadline_ms = min(spec.timeout_ms, budget.remaining_ms())
    if deadline_ms <= 0:
        return exhausted_budget_result(tool_call)
    return transport.call(
        spec,
        args=tool_call.args,
        deadline_ms=deadline_ms,
        idempotency_key=f"agent-{tool_call.id}",
    )

Two properties fall out of those ten lines. Each tool gets the smaller of its own limit and what the run can still afford, so a straggler can't quietly extend a session that should have ended. And the deadline is computed once, before the first attempt — the same discipline as idempotency key generation, and for the same reason: values minted inside a retry loop drift, and drifting values are how duplicates and unbounded waits are born.

Then the deadline should travel with the request. Distributed systems solved this as deadline propagation: an absolute timestamp in a header (Deadline-Epoch-Ms: 1760439300000), read and honored by every hop. Absolute beats relative because a relative timeout re-arms at each service — three hops with thirty-second relative timeouts is a ninety-second operation wearing a thirty-second costume.

Declare the timeout in the tool schema

A model cannot judge latency from a description string. "Runs a report" tells it nothing about whether four seconds is a success or a warning, and the harness has no basis for a per-tool policy beyond guessing. Timeout metadata belongs in the tool schema, next to the name and the argument types:

{
  "name": "export_customers",
  "description": "Runs a filtered customer export and returns a file reference.",
  "timeout_ms": 30000,
  "side_effect_class": "write",
  "schema": { "type": "object", "properties": { "filters": { "type": "object" } } }
}

With timeout_ms and a side-effect class declared, the harness can apply the policy that actually matters per shape of tool:

• Reads get short deadlines and generous retries — cheap to abandon, cheap to repeat.
• Writes get a deadline plus the idempotency key from the moment they're attempted — abandoning is fine only if repeating is deduplicated.
• Long-running operations get neither: a tool that legitimately runs for minutes has a contract bug, and stretching the deadline just hides it. That case gets its own shape, below.

timed_out is a result, not an error

The single highest-leverage change: report the deadline firing as a distinct result state that carries what the model needs to decide. Not status: "error", message: "timeout" — an envelope with meaning:

{
  "status": "timed_out",
  "deadline_ms": 30000,
  "tool": "export_customers",
  "side_effect_status": "unknown",
  "idempotency_key": "agent-call-9f31c2",
  "retryable": true,
  "hint": "Work may still be completing server-side. Retry with the same idempotency key before assuming failure."
}

The load-bearing field is side_effect_status. Its values are none (the request provably never arrived — transport refused before send), possible (sent, outcome unknown), and completed (the server later confirmed the work ran; rare synchronously, common once a ledger exists). After any client-side timeout, the honest default is possible — pretend otherwise and you've built the duplicate machine from the opening scene, with extra steps.

The hint is not decoration. Models are extremely good at following one-line guidance embedded in structured results, and extremely bad at inferring idempotency discipline from vibes. Telling the model "retry the same key" in the envelope converts a class of incidents into a non-event.

Cancellation that actually cancels

Tearing down the socket stops nothing. HTTP servers routinely finish processing requests whose clients vanished; background workers never see the disconnect at all. If you want work to stop, cancellation has to be a server-side behavior, and the only reliable form is cooperative: the operation checks its own deadline at phase boundaries and stops itself.

SAFETY_MARGIN_MS = 2_000

def export_customers(filters, deadline_epoch_ms, run_id):
    batches = plan_scan(filters)                    # phase 1: cheap
    for i, batch in enumerate(batches):
        if now_ms() > deadline_epoch_ms - SAFETY_MARGIN_MS:
            cancel_run(run_id, completed=i, total=len(batches))
            return {"status": "cancelled",
                    "completed_batches": i, "total_batches": len(batches)}
        write_batch(batch)                          # phase 2: the cost
    return finalize(run_id)

The details that make this real rather than decorative: the check sits before each unit of expensive work, so cancellation is cheap by construction; the margin exists because a deadline honored exactly at the wire is a deadline violated half the time; and cancel_run writes a ledger entry — a cancelled run is a fact the system recorded, not an absence. An agent (or an engineer reading the transcript tomorrow) that asks "did the export happen?" should get "cancelled at batch 4 of 90" from the ledger, not a shrug. This is the command-ledger discipline applied to the time axis: abandonment is an event too.

For operations that outlive any single request — the export that takes eleven minutes — cooperative checking inside one HTTP call is still the wrong shape. Cancel needs to be addressable independently of the connection that started the work: a cancel_operation tool, a DELETE /operations/{id}, whatever fits, as long as the agent can reach it without the original socket.

Timeouts and idempotency keys are one mechanism

Read the two contracts together and the ugly state from the opening collapses. The timeout fires with side_effect_status: "possible". The retry — same key, per the envelope's own hint — hits the replay window. Either the key was never inserted (the request never arrived; run the work now) or the stored response comes back (the work completed; here is the result you never saw). "Unknown" resolves into a fact, and the second export never starts.

This is why the retry-after-timeout is the scenario idempotency keys exist for — not the polite 429-and-backoff case, but the ugly one where the answer to "did it run?" is genuinely nobody knows. A stack with only one of the two mechanisms is half-built: deadlines without keys make abandonment safe for the client and dangerous for the system; keys without deadlines make retries correct but let the harness wait forever to learn it should.

Long operations are a contract problem

When a tool legitimately takes minutes, the fix is never timeout_ms: 900000. A synchronous call held open that long fails for reasons that have nothing to do with the work: proxy idle limits, harness memory, a model session paying context for a wait. The operation needs to become asynchronous at the tool boundary, and the agent needs verbs for its lifecycle:

{
  "status": "started",
  "operation_id": "op_7c21",
  "poll_after_ms": 5000,
  "estimated_total_ms": 600000,
  "cancel": {
    "tool": "cancel_operation",
    "args": { "operation_id": "op_7c21" }
  }
}

Polling returns the same honest envelope discipline as everything else in this series: {"status": "running", "completed_batches": 61, "total_batches": 90} while it grinds, and one of succeeded, failed, cancelled, or expired at the end. Note what this shape buys the deadline budget: the call returns in milliseconds, the run's overall wall-clock stays bounded, and the expensive work proceeds server-side where it belongs. The result-side of this contract — how the finished output is sized, paginated, and truncated for a model — is the previous article's subject; this one only insists that the wait be shaped so the model can act.

What cancellation does not fix

The pattern has edges, and honesty about them is what keeps it trustworthy:

• Effects already past your boundary. If phase two of a "cancelled" operation already fired the webhook or sent the email, the ledger says cancelled and the vendor says delivered. Cancellation is cooperative up to the last irreversible step and no further — compensating actions, not cancellation, are the tool for that.
• Partial output mistaken for complete. If a cancelled scan returns the 61 batches it wrote, label them as partial in the envelope. A model that can't tell 61-of-90 from done will confidently build on the wrong number.
• A lying envelope. side_effect_status: "none" stamped unconditionally by a lazy harness is worse than no field: it teaches the model that abandonment is always free. Default to possible, upgrade to better only when you can prove it.
• User patience. Budgets make waiting bounded and observable; they do not make it shorter. The run that needs eleven minutes of tool time still needs a product decision, not a timeout tune.

Testing: a tool that hangs

Every one of these contracts is testable with a fake clock or short deadlines, and the tests belong next to the tool's own:

def test_deadline_fires_with_honest_envelope(harness):
    harness.register("slow_search", handler=sleep_5s, timeout_ms=50)
    result = harness.call_and_await("slow_search", {"q": "x"})
    assert result["status"] == "timed_out"
    assert result["side_effect_status"] in ("none", "possible")
    assert result["deadline_ms"] == 50
    assert harness.no_transport_alive()          # no zombie connections

def test_retry_after_timeout_replays_not_repeats(harness, proxy):
    proxy.drop_response_after_commit("export_customers")   # fault injection
    r1 = harness.call_and_await("export_customers", filters)
    assert r1["status"] == "timed_out"
    r2 = harness.retry(r1)                       # same idempotency key
    assert r2["status"] == "succeeded"
    assert harness.server_side_runs("export_customers") == 1

The second test is the one that catches real regressions, and it needs the fault-injecting proxy from the idempotency piece — the transport must be made to lie (commit, then drop the reply) because that lie is the whole scenario. A third test exercises cooperative cancellation: a handler that checks its deadline mid-loop must stop, write its cancelled ledger entry, and release its resources, with the entry asserting the batch counts it reported.

The checklist

• One deadline per call, derived as the minimum of the tool's declared limit and the run's remaining budget — never a fresh fixed slice per attempt.
• Deadlines travel with the request as absolute timestamps and are honored at phase boundaries inside the operation.
• timed_out is a first-class result with side_effect_status and the idempotency key attached — never a generic error.
• Cancelled operations write ledger entries with progress counts; abandonment is an event, not an absence.
• Retries after timeouts reuse the key, so the replay window — not the model's guesswork — resolves whether the work happened.
• Operations longer than a few seconds become start/poll/cancel jobs; a synchronous multi-minute wait is the bug.
• Hanging-tool tests in CI assert envelope shape, exactly-one server-side execution, and no zombie transports.

An agent that can stop waiting without being lied to makes strictly better decisions: it treats a deadline as a scheduling fact instead of a verdict. Deadlines, cancellation records, and one honest envelope are how you buy that — for the price of a timestamp.