Tool Response Design: Giving Agents Results They Can Actually Use

An agent asks list_tickets for the open queue. The tool faithfully returns 1,847 tickets as JSON — 900 KB of it. The harness drops it into the transcript. Three turns later the model is re-reading ticket subjects from four turns ago because half its context is one tool response it cannot forget. Or the other variant: the harness, defending itself, clips the response to the last 8,000 tokens — mid-array, mid-object, mid-string. The model now sees syntactically broken JSON that ends in the middle of a ticket summary, and it proceeds as if it had seen the whole queue. Both failures come from the same root: nobody designed the response. The tool returned what was easy to return, and the context window paid for it.

Tool schemas get all the design attention — names, enums, descriptions. But the response is the half the model actually reads. Every byte a tool returns becomes part of the prompt for the rest of the session: it cannot be un-sent, it competes with everything else for attention, and if it is silently mangled the model has no way to know. The response is not a return value. It is a prompt you write with data.

That reframing gives us the whole design space, and this article walks it: budgets enforced at the source, truncation that respects structure and says so, cursors that survive shifting data, shapes sized to the next decision, and errors the model can act on.

Budget the response at the source

The budget belongs in the tool contract, not in the transport. A harness-level clip is the worst possible place to enforce a limit: it happens after serialization, it knows nothing about the data's structure, and it is invisible to both the tool and the model. Enforcing the budget where the data is assembled lets you cut cleanly and say what you cut.

Make the budget a declared parameter so the model can participate:

{
  "name": "list_tickets",
  "description": "List tickets in the queue. Returns at most `limit` tickets; if more exist, `next_cursor` is set.",
  "parameters": {
    "status": { "type": "string", "enum": ["open", "in_progress", "closed"] },
    "limit": { "type": "integer", "maximum": 50, "default": 20 },
    "fields": { "type": "array", "items": { "enum": ["id", "subject", "priority", "assignee", "updated_at"] } }
  }
}

Three budgets live here, and each is a real control. limit caps item count. fields caps per-item width — most decisions need identity and a few salient attributes, not the full record. And the server enforces a byte ceiling regardless, because the model should not be able to ask for something that poisons its own context. A tool whose permissive defaults can return megabytes is a tool that will, one day, be called with those defaults at turn forty of a long session.

The harness clip still exists as a backstop — transport defenses are fine. What's not fine is a system whose only output budget is a clip the model can't see.

Truncate on item boundaries, never mid-structure

When the result exceeds the budget, cut where the data has seams. A JSON array of records has a natural unit — the record. A response clipped mid-object forces the model to either hallucinate the missing tail or waste a turn on a repair call. A response cut between records is complete data about fewer things, and the model can reason about exactly what it has.

{
  "tickets": [
    { "id": "TCK-8123", "subject": "Refund duplicated for order 8123", "priority": "high", "updated_at": "2026-10-11T09:14:00Z" },
    { "id": "TCK-8124", "subject": "Export job stuck at 90%", "priority": "medium", "updated_at": "2026-10-11T08:02:00Z" }
  ],
  "total_matching": 1847,
  "returned": 2,
  "truncated": true,
  "next_cursor": "eyJzIjoib3BlbiIsImsiOlsiVDhJMTI0Il19"
}

The envelope carries four facts the model needs: what came back (returned), what exists (total_matching), that this is not everything (truncated), and how to get the next page (next_cursor). total_matching deserves emphasis — without it, "two tickets" and "two of 1,847 tickets" look identical, and an agent that concludes the queue is nearly empty will make confidently wrong downstream decisions.

The same principle applies inside a single large record. A get_ticket response with 200 comments should truncate the comment list — "comments_returned": 20, "comments_total": 200 — not the ticket, and never the whole payload.

The truncation marker must survive the trip

Here is the failure that does the real damage. The tool truncates honestly, sets truncated: true, and returns 30 KB. The harness — defending a per-result cap — clips that response to 8,000 tokens. The clip lands after the envelope's opening fields, and the truncated: true line is gone. The model receives complete-looking JSON (the clip happened to end at an item boundary by luck) with no marker, and no next_cursor. It believes it has seen everything.

Silent truncation is worse than no truncation, because it converts a size problem into a correctness problem. The defenses are structural. Put the envelope's status fields at the top of the response, before the payload — the first thing any clipper keeps — and repeat the essentials at the bottom as a footer, so both a head-clip and a tail-clip leave a trace. And make the harness clip loudly: when it must cut, it should splice in an explicit sentinel like [RESULT TRUNCATED BY HARNESS AT 8192 TOKENS — REQUEST WITH NARROWER PARAMETERS] rather than cutting cleanly enough to look whole. A visible seam the model can notice beats an invisible one it can't.

The round-trip requirement makes this testable: every truncation signal the source emits must still be detectable after the transport's worst-case munging. If it isn't, the design has a silent-truncation path, and that path will execute eventually.

Cursors that don't lie when the data moves

Once results can be truncated, the model will page. Offset pagination — ?page=3 — is the familiar default and the wrong one for agents. The queue is live: while the model works through pages, tickets get closed, new ones arrive, and an offset into a shifting list silently skips or repeats records. The model pages through what it believes is a stable snapshot and receives three different snapshots stapled together.

A cursor encodes position in the sort order, not a row number:

def encode_cursor(sort_key, exclusive_ids):
    # Position = last item's sort value + tie-breaker, opaque to the model.
    # Server-side it's just (updated_at, id) — stable under inserts elsewhere.
    return base64(json.dumps({"sk": sort_key.isoformat(), "xid": exclusive_ids}))

rows = db.execute(
    "SELECT ... FROM tickets WHERE status = %s "
    "AND (updated_at, id) < (%s, %s) ORDER BY updated_at DESC, id DESC LIMIT %s",
    (status, cursor.sk, cursor.xid, limit + 1),
)
has_more = len(rows) > limit

Keyset pagination costs nothing extra, survives concurrent writes, and gives the model a next_cursor that means the same thing no matter when it's used. The cursor should be opaque — base64 over JSON is fine — because an opaque token signals "pass this back verbatim" far better than a raw offset, which invites the model to do math on it. Models will improvise with anything that looks like a number; hand them a token whose only documented operation is "return as-is."

One honesty rule: cursors expire. If the cursor's snapshot basis (say, a sort field that got rewritten) is gone, return an explicit error — cursor_expired, restart from the beginning — never quietly degrade into an offset-style guess.

Shape the response for the next decision

The model called the tool to make one decision. The response should contain what that decision needs, and nothing that doesn't change it. This is prompt engineering with data.

In practice that means three shapes, deliberately separated:

• List responses return identity and discriminating fields — id, subject, priority, timestamps. Not full bodies, not attachments, not change history. The list's job is selection.
• Detail responses fetch one selected record, still projected: a get_ticket that returns the ticket plus a bounded, cursored view of its comments, not the comment firehose.
• References instead of blobs — return a path, a URL, or a handle for anything large. If the next step genuinely needs the 40 KB attachment, there's a tool for that, and the model can call it once for the one attachment that matters instead of paying for all fourteen on every list call.

Watch for payload fields that exist because the underlying API returns them, not because any decision consumes them. Internal IDs of external systems, audit columns, denormalized copies of parent records — each one is context spent on nothing. Every field in a tool response should be able to answer the question "which next action would change if this value changed?" Fields with no answer don't belong in the envelope.

And keep it machine-shaped. A tool that returns a pretty-printed Markdown table has forced the model to re-parse presentation to recover data. JSON (or the structured format your harness uses natively) with short, consistent field names is both smaller and harder to misread.

Errors are responses too

The error path is where response design usually collapses: the tool throws, the harness catches, and the model receives a stringified stack trace or a bare Error: ECONNRESET. That's a prompt too — an expensive, useless one.

Errors deserve the same envelope discipline as successes, because the model's next move depends on being able to classify the failure:

{
  "error": {
    "code": "rate_limited",
    "retryable": true,
    "retry_after_seconds": 30,
    "detail": "Ticket API quota exceeded for this key.",
    "ref": "err_01JDYZ3W7Q"
  }
}

Four fields carry the decision. code is a closed enum the model can branch on — not a message to be interpreted. retryable plus retry_after_seconds tells a retrying agent what to do without guessing; a non-retryable error steers it toward a different tool or a report to the user instead of three more identical attempts. ref is the durable error id an engineer can grep for when the transcript shows err_01JDYZ3W7Q at 14:02. Never let a raw exception reach the model — a stack trace is context poison with no decision value, and it often leaks paths, hosts, and keys.

This is also where the series connects: a retried call under an idempotency key replays the stored response, error included, so error envelopes must be stable enough to replay honestly. And a rate-limited call fanned out in parallel with its siblings should fail with retryable: true, not cascade into a confusing mix of clipped successes.

Test the envelope, not just the endpoint

Response contracts regress quietly: someone adds a field, the ORM starts returning an extra column, a vendor payload grows. The tests that hold the line assert envelope invariants directly:

def test_list_response_under_budget(fake_queue_with(2000, tickets)):
    r = tools.list_tickets(status="open")
    assert serialized_size(r) < MAX_RESPONSE_BYTES      # the budget is real

def test_truncation_marker_survives_clip(fake_queue_with(2000, tickets)):
    r = tools.list_tickets(status="open")
    clipped = harness_clip(r, tokens=8192)              # worst-case transport
    assert "truncated" in visible_fields(clipped)       # marker still detectable

def test_cursor_walks_all_rows_once(fake_queue_with(2000, tickets)):
    seen, cursor = [], None
    while True:
        r = tools.list_tickets(status="open", cursor=cursor)
        seen += [t["id"] for t in r["tickets"]]
        if not r.get("next_cursor"): break
        cursor = r["next_cursor"]
    assert sorted(seen) == sorted(all_ids)              # no loss, no duplicates

The third test is the one that catches the offset-pagination trap under concurrent writes — inject a row between pages in the fixture and an offset scheme fails it immediately. The second test encodes this article's core claim: it fails the moment someone adds a silent truncation path.

The checklist

• Enforce output budgets at the source — item count, projected fields, byte ceiling — never only as a harness clip.
• Cut on item boundaries; return returned, total_matching, truncated, and next_cursor in one envelope.
• Put truncation markers where no clip can remove them, and make harness clips splice a visible sentinel rather than cut cleanly.
• Page by opaque keyset cursor, not offset; fail expired cursors loudly.
• Three shapes, three jobs: list for selection, detail for one record, references for anything large.
• Errors in the same envelope discipline: closed code enum, retryable, retry_after_seconds, durable ref — never stack traces.
• CI asserts the envelope: size under budget, marker survives clipping, cursor walks the full set exactly once.

A tool response is the only thing the model learns from a call. Design it like a prompt — bounded, structured, honest about what's missing — and the model's next turn starts from facts instead of fragments.