An agent does not send one long prompt. It sends the whole conversation again on every turn. We measured six runs of the same audit-report task (Claude Opus · Sonnet and DeepSeek-V4.1-Flash-1017 on our DGX Spark ×4 cluster) and found that what decides time and cost is how many times the conversation is re-read, and whether that re-read hits the prefix cache.
Two numbers that look alike
- Session size: how long the conversation is when the agent finishes (last turn’s context + last output).
- Cumulative prefill: the sum of the context read on every turn. This is what the server processes and what an API bills.
| Run | Turns | Context at start(*1) | Session size | Added by the work | Cumulative prefill | Prefill ÷ size |
|---|---|---|---|---|---|---|
| Sonnet medium | 35 | 50k | 139k | 88k | 3.65M | 26× |
| Opus medium | 59 | 50k | 170k | 120k | 7.13M | 42× |
| Sonnet high | 68 | 50k | 167k | 117k | 7.89M | 47× |
| DeepSeek low | 80 | 28k | 153k | 125k | 7.98M | 52× |
| DeepSeek high | 136 | 28k | 178k | 150k | 14.08M | 79× |
| DeepSeek max | 150 | 28k | 238k | 210k | 20.62M | 87× |
(*1) System prompt, tool definitions and the task prompt. Claude Code ships a different system prompt to Claude than to an external model. MCP was off in all runs. Cumulative prefill includes the footing sub-agents. No run compacted its context.
Session size varies 1.7× between runs. Cumulative prefill varies 5.6×. Turn count, not context size, sets the cost.
The cache does almost all the work
On the DeepSeek low run, 97% of the cumulative prefill was a cache hit.
| DeepSeek low | Tokens | At 6,200 tok/s prefill |
|---|---|---|
| Cumulative prefill | 7.98M | about 21 min, longer than the 15-min run |
| Cache hits | 7.73M | close to zero |
| Actually prefilled | 185k | about 30 s |
Without the cache this run would take well over 35 minutes. We have seen it happen: an early run where an engine bug disabled the prefix cache took 107 minutes. Its output (110k tokens at about 68 tok/s) explains roughly 27 of them; re-prefilling 24M input tokens explains most of the rest.
The same structure holds for API billing. Cache reads are cheap, which is why Sonnet medium read 3.65M tokens and cost USD 1.31.
What breaks the cache
The prefix must match token for token. Things we measured:
- Effort level. Our server puts the reasoning effort on the first line of the prompt. A branch forked from a 224k-token session hit 99.9% of the cache at the same effort and 0% at a different one (38 s of prefill).
- Tool list order. Headless Claude Code sent its first requests before MCP servers finished connecting, so the tool list
differed from request to request (23 → 151 tools, three resource tools appearing late in the middle of the list). Forked
branches missed the cache by that difference. Turning MCP off (
--strict-mcp-config) fixed it. - Sub-agents. A sub-agent starts with its own system prompt and cannot reuse the parent’s cache. A forked session
(
--resume --fork-session) with identical settings can.
On a local server, prefill stops everyone
With chunked prefill, while one request prefills, decoding for the other requests nearly stops. A sub-agent reading 50k fresh tokens pauses the main agent and every other branch for about 8 seconds. This is one reason splitting the work into parallel branches made DeepSeek slower (27 → 39 min) while it made Sonnet 24% faster: per-stream decode also fell to about 27 tok/s when the branches shared one server.
Takeaways
- An agent’s cost is turns × context, not context. Fewer turns beat a shorter prompt.
- Keep the prefix stable: same effort, same tool list, same system prompt. One changed line at the top re-prefills everything.
- Measure cache reads, not input tokens. Our server’s
input_tokensis not reliable;cache_read_input_tokensis. - Tools that remove thinking (a summary view, prefilled values, a channel to ask the user) cut turns, and that is where time goes.
- On a single local server, avoid parallel sub-agents that each prefill fresh context.
We will re-measure as the server stack and Claude Code change.