OpenAI Codex · Tool loop

The stream is still going. Tools already started.

The model is still typing. The sound of reading a file already started. Sampling’s timing is: pin futures inside the stream, drain after it. Persist first, then wait for results.

Course goalAfter this lesson you can explain three things. First: which frame the parser decides a tool starts. Second: why a request is written to history before it is pinned on the queue. Third: on a clean end, an early close, or Esc — who finishes tools already running, and what history keeps.
Play first · Drag the progress. Watch which frame starts the run.
Same SSE stream: drag the progress, watch when tools start
Exit
Drag the slider or tap an event cell. Left runs inside the stream. Right waits for the stream to end. The exit switch changes how the last frame wraps up.
Run inside the stream0 persisted
Waiting to start.
Wait for the stream0 persisted
Waiting to start.
Logic trail · each animation step maps to a source span
  1. An SSE frame first becomes a generic event, no business meaning yetresponses.rs L164
  2. kind output_item.done yields OutputItemDoneresponses.rs L352
  3. The sampling loop hands it to handle_output_item_done immediatelyturn.rs L2384
  4. Write the function_call to history and rollout firststream_events_utils.rs L316
  5. Then pin the tool and push the ordered queuestream_events_utils.rs L320
  6. Stream end, break, or cancel only leaves the receive loopturn.rs L2282
  7. drain writes results to history in insert orderturn.rs L2135
  8. Only then check the cancel token; Stream is retryableturn.rs L2760
Drag the progress or hit Play. Watch which frame the parser decides a tool starts.
The start lineThe left stamps and starts on the first OutputItemDone. The right waits for the last frame. While the model is still talking, the two sides already split.
Broken streamThe left keeps persisted requests and drained results. A retry reads that history. The right never wrote the request. A retry starts from empty history.
EscThe left keeps the request, writes abort copy, marks TurnAborted, and does not fire another round. The right wrote nothing.
Teaching sketch: the event rail and timings are course fixtures, to contrast two clocks. Line numbers on the trail match openai/codex commit 4f39251a01.
Idea 1 · Stamp the request first. Then start the tool.
What problem it solves

You ask the model to read three files and write a summary. The screen is still typing. The sound of reading a file already started. Then you hit Esc. The UI stops. History still keeps that request, sometimes the result too. You thought cancel meant nothing happened. The runtime does not book it that way.

The more common case: the model already emitted two function_calls, a third is still on the way, and SSE closes before response.completed. Should the next retry see empty history, or calls and results already persisted?

Wait for Completed to write, and an early close throws away calls that were already complete. A retry makes the model emit the same calls again. Skip the write on cancel and history keeps a half request. Model and UI both see an unclosed call.

What the idea is

OutputItemDone is the parse layer folding one response.output_item.done frame into a business event. The moment it arrives, sampling writes it to session history and rollout, then wraps tool exec as a future on the ordered queue. Cancel uses a child token: when the parent lights, this tool stops with it. However fast cancel arrives, this function_call is already in history. At most you write one more aborted by user.

Source: codex-rs/core/src/stream_events_utils.rs lines 190–192; lines 316–327. The comment above the type alias locks the contract: finished model output is recorded immediately. If the turn later cancels, history and rollout stay in sync.

SSE byte stream Still emitting frames OutputItemDone One complete tool request Write history function_call persists first Pin a future The tool is already running Later delta Still on the way Input is a finished tool request. Output is a receipt: history grew one row, the queue grew one still-running future. The model has not finished talking. The request is already stamped.
Teaching diagram: In the same frame: persist first, then pin. Later typing overlaps tool time.
Why it lasts

History only appends. It does not rewrite. The tool already read disk. That fact happened. A cancel tree can interrupt exec. It cannot unwrite a request already on disk. Write the request first, the result second, and the transcript stays closed. The contract is not Rust-specific. In TypeScript it is still append then push a promise.

Source: AGENTS.md lines 91–100, Model visible context, first rule: No history rewrite.

Idea 2 · Pin inside the stream. Drain at every exit.
What problem it solves

The model often emits a file read, then keeps writing an explanation. Wait for the closing whistle to start work, and you serialize file-read latency with typing latency. Starting inside the stream overlaps those two. The cost: cancel and a broken stream must claim futures already running. No owner, and you get orphans — tools still running, history out of sync.

What the idea is

The receive loop — clean Completed, early close, or or_cancel — only leaves the loop. The function has not returned. It always calls drain_in_flight, waits for each future in insert order, writes history, and only then checks the cancel token.

A broken stream takes Stream and is retryable. A retry rebuilds the prompt from clone_history; calls and results already written stay. Esc takes TurnAborted and is not retryable. Running tools write abort copy. drain writes that as ordinary output.

Source: codex-rs/core/src/session/turn.rs lines 2282–2284; lines 2744–2762. codex-rs/protocol/src/error.rs lines 88–93, lines 364–390.

Receive loop Completed Close the stream early Esc drain_in_flight Write results in insert order, then check the cancel token
Teaching diagram: Three roads meet at drain first, then split into follow-up, retry, or abort.
Pin futures inside the stream. Drain after it, together.
Why it lasts

Start mid-stream and every exit must wait for the set. Ownership stays in the sampling function’s locals. There is no second background reaper. Success, error, and cancel share this wrap-up. Same in another language: leave the async loop, allSettled first, then decide retry or abort.

Idea 3 · Exec may run in parallel. History writes in emit order.

Three tools may run at once. Whoever finishes first, history still writes results in the order the model emitted them. Write in completion order and the same session replayed twice may diverge; the prompt cache gets more brittle too. The ordered queue splits observation order from exec order. The concurrency gate lives on another layer. Here, just this: pin order is later drain order.

Source: codex-rs/core/src/session/turn.rs lines 2130–2154; lines 2391–2397.

Side-by-side · another answer to the same question

Claude Code: wait for the stream by default, plus one in-stream gate

On the default path the stream loop only collects tool_use, and runTools starts after for await ends. A broken stream just drops collected blocks. You skip local ownership that drains at every exit. The cost: tool latency and typing latency serialize.

When streamingToolExecution is on, behavior moves toward Codex: addTool inside the stream, start immediately. Failure rollback must discard tools already running so old ids do not leak into a retry. Codex has no matching discard, because it persists first and retries read history.

Source-checked on both sides · query.ts lines 551–568, lines 1380–1382

DSH: a three-stage waterfall plus a monotone Guard — who may refuse

DSH’s entry is an already-formed tool call. pre / guard / around / post answer who may refuse, and whether a result remains after refusal. Guard has a refuse reason or an abstain — no “allow” option. Its drained is wrap-up inside one execute. While SSE is still in flight, this waterfall has not started.

The words look close. The exits differ. One guards monotone permission. One guards a closed streaming transcript. Move Guard into Codex and it will not stop a broken stream from dropping transcript. Move persist-then-drain into DSH and it will not answer whether a plugin can turn refuse into allow.

Source-checked on both sides · tools/src/index.ts lines 1–4, lines 703–711, lines 1328–1337 · DSH · three-stage waterfall and monotone Guard
Classroom Exercise
01

Swap write and pin

In the tool branch of handle_output_item_done, swap record_completed_response_item and Box::pin(handle_tool_call). Cancel happens before pin, before persist. What do the next sample and session restore see?

Pin the answer on history only appending, and on when drain_in_flight writes.

Takeaway: On OutputItemDone, write first, then start. Every stream exit drains first. Results write in emit order. Cancel interrupts exec. The transcript already written stays.