Backpressure and Rate Limits: Making Agent Tool Calls Safe to Fan Out
Learn how to make AI agent tool calls safe to fan out: client-side token buckets and concurrency caps per tool, honoring Retry-After and rate-limit headers, 429s surfaced to the model as structured rate_limited results with a retry hint instead of generic errors, deadline-bounded backoff with full jitter, a bounded executor queue that sheds load honestly, single-flight coalescing of the fan-out, and a throttling-proxy test you can run in CI.