Replayable Test Harnesses for AI Agents

Introduction

Tool-using AI agents are difficult to test because the interesting behavior crosses several boundaries at once. The model may choose a tool, the runtime may validate arguments, the tool may call an external API, and the workflow may persist state before deciding what to do next. A normal unit test can cover one function, but it rarely proves that the whole agent survives a timeout, resumes after a crash, avoids duplicate writes, and explains why it stopped.

The answer is not to hope the model makes the same decision every time. The useful pattern is a replayable test harness: record the runtime boundary during a real or scripted run, then replay that boundary deterministically with fake clocks, fake tools, and deliberate failures. The model can remain probabilistic in production while the workflow contract stays testable in CI.

This article shows how to build that harness for AI agent workflows. The examples use TypeScript, but the same design works in queues, workflow engines, serverless jobs, background workers, and durable agent runtimes.

Test the Runtime Boundary, Not the Prompt

Prompt snapshots are a weak test surface. They are useful for review, but they do not tell you whether the runtime protected a write tool, persisted a checkpoint, honored a budget, or resumed without duplicating work. A replayable harness should wrap the boundary where nondeterministic decisions become operational events.

For most agents, that boundary has four parts:

• Model calls, including the normalized request and the chosen response.
• Tool calls, including the validated arguments and returned result.
• State writes, including checkpoints, command records, approvals, and terminal status.
• Time, including deadlines, leases, retry delays, and scheduled wakeups.

The production runtime and the test harness should use the same interface. That keeps replay from becoming a parallel implementation that silently drifts away from reality.

type ModelRequest = {
  runId: string;
  stepId: string;
  messages: Array<{ role: "system" | "user" | "assistant"; content: string }>;
  tools: Array<{ name: string; schema: unknown }>;
};

type ModelResponse = {
  message: string;
  toolCall?: {
    name: string;
    arguments: Record<string, unknown>;
  };
};

type ToolRequest = {
  runId: string;
  stepId: string;
  toolName: string;
  arguments: Record<string, unknown>;
  idempotencyKey: string;
};

type AgentIO = {
  now(): Date;
  callModel(request: ModelRequest): Promise<ModelResponse>;
  callTool(request: ToolRequest): Promise<unknown>;
  writeEvent(event: AgentEvent): Promise<void>;
};

The agent loop depends on AgentIO, not on a concrete model SDK, HTTP client, or wall clock. Production code passes real adapters. Tests pass a recorder, a replayer, or a fault-injecting wrapper. That single seam is what makes the workflow inspectable.

Keep validation outside the model

The model can propose a tool call, but validation should live in deterministic code before the call becomes executable. Tests should assert that invalid proposals become rejected events, not that the model never makes them. This distinction matters because a prompt change can improve suggestions, but only runtime policy can guarantee safety.

Record Canonical Traces

A trace is not a raw log dump. It is the minimum contract needed to replay the run and prove the same runtime decisions still happen. Store entries in a stable order, remove volatile fields, normalize arguments, and keep enough metadata to diagnose failures later.

type TraceEntry =
  | {
      kind: "model";
      stepId: string;
      requestHash: string;
      response: ModelResponse;
    }
  | {
      kind: "tool";
      stepId: string;
      toolName: string;
      argumentHash: string;
      idempotencyKey: string;
      result: unknown;
    }
  | {
      kind: "event";
      stepId: string;
      eventType: string;
      payload: Record<string, unknown>;
    }
  | {
      kind: "clock";
      label: string;
      isoTime: string;
    };

function stableJson(value: unknown): string {
  if (Array.isArray(value)) {
    return `[${value.map(stableJson).join(",")}]`;
  }

  if (value && typeof value === "object") {
    return `{${Object.entries(value as Record<string, unknown>)
      .sort(([left], [right]) => left.localeCompare(right))
      .map(([key, nested]) => `${JSON.stringify(key)}:${stableJson(nested)}`)
      .join(",")}}`;
  }

  return JSON.stringify(value);
}

async function recordToolCall(
  trace: TraceWriter,
  request: ToolRequest,
  callRealTool: () => Promise<unknown>,
) {
  const result = await callRealTool();

  await trace.append({
    kind: "tool",
    stepId: request.stepId,
    toolName: request.toolName,
    argumentHash: sha256(stableJson(request.arguments)),
    idempotencyKey: request.idempotencyKey,
    result,
  });

  return result;
}

Hashes keep traces compact while still detecting drift. If a tool argument changes, the replay fails at the exact boundary instead of continuing with a stale fixture. When debugging, store the full normalized request in a private artifact, but keep public or committed traces free of secrets and customer data.

Version the trace format

Trace files should include a schema version and runtime version. Agent systems change quickly: a new tool argument, a renamed event, or a stricter policy can make an old trace fail for a legitimate reason. Versioning lets the test harness distinguish "the workflow regressed" from "this fixture needs migration."

Replay Without Real Side Effects

Replay mode should make the agent believe it is running normally while every external boundary is served from the trace. That means model responses come from recorded entries, tools return recorded results, and time advances according to recorded clock entries or an injected fake clock.

class TraceReplayer implements AgentIO {
  private cursor = 0;

  constructor(
    private readonly entries: TraceEntry[],
    private readonly events: AgentEvent[] = [],
  ) {}

  now(): Date {
    const entry = this.next("clock");
    return new Date(entry.isoTime);
  }

  async callModel(request: ModelRequest): Promise<ModelResponse> {
    const entry = this.next("model");
    const actualHash = sha256(stableJson(request));

    if (entry.requestHash !== actualHash) {
      throw new Error(
        `model request drift at ${request.stepId}: expected ${entry.requestHash}, got ${actualHash}`,
      );
    }

    return entry.response;
  }

  async callTool(request: ToolRequest): Promise<unknown> {
    const entry = this.next("tool");
    const actualHash = sha256(stableJson(request.arguments));

    if (entry.toolName !== request.toolName) {
      throw new Error(`tool drift: expected ${entry.toolName}, got ${request.toolName}`);
    }
    if (entry.argumentHash !== actualHash) {
      throw new Error(`argument drift for ${request.toolName}`);
    }
    if (entry.idempotencyKey !== request.idempotencyKey) {
      throw new Error(`idempotency key drift for ${request.toolName}`);
    }

    return entry.result;
  }

  async writeEvent(event: AgentEvent): Promise<void> {
    this.events.push(event);
  }

  private next<TKind extends TraceEntry["kind"]>(
    kind: TKind,
  ): Extract<TraceEntry, { kind: TKind }> {
    const entry = this.entries[this.cursor++];
    if (!entry || entry.kind !== kind) {
      throw new Error(`trace drift: expected ${kind}, got ${entry?.kind ?? "end"}`);
    }
    return entry as Extract<TraceEntry, { kind: TKind }>;
  }
}

The strict cursor is deliberate. If the workflow adds an extra model call, skips a policy event, or calls a different tool, the replay fails immediately. That is the point: an agent test should catch control-flow drift before a changed prompt or refactor reaches production.

Do not assert on every token

Replay tests should be strict at operational boundaries and flexible around explanatory text. Exact model prose is usually the wrong invariant. Better assertions include the selected tool, normalized arguments, state transition, command key, approval requirement, idempotency key, and terminal status.

Inject the Failures You Fear

A passing replay proves the happy path still works. A useful harness also mutates the boundary so the workflow experiences production-shaped failures: model refusal, malformed tool arguments, tool timeout, partial success, process crash after a side effect, expired approval, exhausted budget, or duplicate worker execution.

Do this with small wrappers around AgentIO, not with sleeps or real external dependencies.

type Fault =
  | { at: "model"; stepId: string; error: Error }
  | { at: "tool"; toolName: string; error: Error }
  | { at: "afterEvent"; eventType: string; error: Error };

class FaultInjectingIO implements AgentIO {
  constructor(
    private readonly inner: AgentIO,
    private readonly faults: Fault[],
  ) {}

  now() {
    return this.inner.now();
  }

  async callModel(request: ModelRequest) {
    const fault = this.faults.find(
      (candidate) => candidate.at === "model" && candidate.stepId === request.stepId,
    );
    if (fault?.at === "model") throw fault.error;
    return this.inner.callModel(request);
  }

  async callTool(request: ToolRequest) {
    const fault = this.faults.find(
      (candidate) => candidate.at === "tool" && candidate.toolName === request.toolName,
    );
    if (fault?.at === "tool") throw fault.error;
    return this.inner.callTool(request);
  }

  async writeEvent(event: AgentEvent) {
    await this.inner.writeEvent(event);
    const fault = this.faults.find(
      (candidate) => candidate.at === "afterEvent" && candidate.eventType === event.type,
    );
    if (fault?.at === "afterEvent") throw fault.error;
  }
}

The afterEvent fault is especially valuable. It simulates the uncomfortable production case where the runtime persisted intent or called a tool, then crashed before advancing the workflow. Recovery should reload durable state, observe that the event already happened, and continue without replaying unsafe work.

Test recovery as a two-run scenario

Crash tests should usually run the workflow twice. The first run injects the failure. The second run starts from the persisted database state and uses the same idempotency keys. The assertion is not "the first run returned success." The assertion is that the combined result is correct after recovery.

it("resumes after a crash without sending a duplicate message", async () => {
  const db = await createTestDatabase();
  const trace = await loadTrace("approve-and-notify.json");

  const firstIO = new FaultInjectingIO(new TraceReplayer(trace), [
    {
      at: "afterEvent",
      eventType: "tool_succeeded",
      error: new Error("simulated worker crash"),
    },
  ]);

  await expect(runAgentWorkflow(db, firstIO)).rejects.toThrow("simulated worker crash");

  const secondIO = new TraceReplayer(trace);
  await runAgentWorkflow(db, secondIO);

  const sends = await db.messages.findByRunId("run_123");
  expect(sends).toHaveLength(1);
  expect(sends[0].idempotency_key).toBe("run_123:notify-user:approved");

  const run = await db.agentRuns.findById("run_123");
  expect(run.status).toBe("completed");
});

This is the test that catches the expensive class of agent bugs: duplicate comments, duplicate emails, duplicate tickets, repeated deployments, and workflows that get stuck in an uncertain state after restart.

Decide What Makes a Trace Pass

Replay without clear assertions becomes a snapshot test with a nicer name. Before adding a trace to CI, decide which invariants the trace is meant to protect. A small number of explicit assertions is usually better than comparing a giant JSON blob.

Useful invariants include:

• Every write tool has a stable idempotency key.
• Approval-required tools cannot execute before an approval event exists.
• A crash after recording a side effect does not create a second side effect.
• A timeout moves the run to a retryable or blocked state, not silent success.
• Budget exhaustion stops the workflow at the next safe boundary.
• Reconciliation is attempted before retrying an uncertain write.
• Final events contain enough context for support and observability.

Separate these from quality evaluations. A replay harness tells you whether the workflow is reliable. It does not prove the model made the best plan, wrote the clearest summary, or selected the highest-quality patch. Those checks belong in evals, code review, or product-specific acceptance tests.

Keep traces small and named by risk

Trace names should describe the production risk they protect: crash-after-ticket-create, approval-expired-before-deploy, tool-timeout-before-checkpoint, or budget-exhausted-during-research. Avoid a single mega-trace that tries to cover everything. Small traces fail with better error messages and are easier to update when the workflow evolves.

Conclusion and Next Steps

Reliable agents need more than prompt tests. They need deterministic pressure around the runtime boundary where model output becomes state, tool calls, approvals, retries, and side effects. A replayable harness gives you that pressure without depending on live model calls or real external systems.

Start with one risky workflow and record a happy-path trace. Add strict checks for tool arguments, idempotency keys, and state transitions. Then add two failure traces: one timeout before a tool returns and one crash after a side effect is recorded. Once those tests run in CI, every prompt change and runtime refactor has to preserve the recovery contract before it reaches users.