Does telling an agent to think harder give a better result? We ran the same task three times on DeepSeek-V4.1-Flash-1017 on our DGX Spark ×4 cluster, changing only the effort level. The score did not move, the time doubled, and the way the agent worked changed.
How we measured
- Task: roll an audit report forward one year (last year’s report + this year’s trial balance → this year’s draft).
- Harness: Claude Code headless, same prompt, same skill, same scripted answers from the “team member”.
- One run at a time, so the runs did not share the server.
- Effort is passed by Claude Code; the server puts it on the first line of the prompt.
Same output
| Effort | Time (min) | Turns | Output tokens | Code (lines) | Score: statements · all R / P | To confirm | Footing checks · tables |
|---|---|---|---|---|---|---|---|
| low | 15.0 | 80 | 47k | 152 | 0.96 / 1.00 · 0.83 / 0.80 | 44 | 202 · 29 |
| high | 20.7 | 136 | 60k | 317 | same | 44 | 208 · 29 |
| max | 30.3 | 150 | 97k | 154 | same | 45 | 219 · 30 |
The skill’s roll-forward command and the team member’s answers decide the numbers, so there is little room for effort to reach the score. The one difference is footing: max footed one more table and set 17 more checks. All three found zero mismatches.
Same speed, more tokens
Per-response output speed is 81.3, 82.3 and 82.2 tok/s. Effort changes how much the model writes, not how fast. Max took 30 minutes because it wrote twice as much as low, not because it slowed down.
Where the time went
| Phase (min) | low | high | max |
|---|---|---|---|
| Reading the source before asking | 3.6 | 0.9 | 2.4 |
| Asking 21 questions and applying answers | 4.1 | 9.4 | 12.2 |
| Footing sub-agents | 3.0 | 4.2 | 6.6 |
| Self-check after review | 2.2 | 3.9 | 5.5 |
| Final report | 0.7 | 0.7 | 1.7 |
- The biggest gap is the question-and-apply phase.
- Sub-agents inherit the parent’s effort. On the same batch of 12 tables, low used 27k characters of reasoning and 178 s; max used 63k and 387 s.
- Max spent 17k characters of reasoning on the final report alone.
A different way of working
All three got the same instruction: ask one question at a time and apply each answer right away.
- Low: look at everything first, then finish in one go. It read the source ahead, asked all 21 questions in 2.4 minutes, and applied them in two scripts. It chose speed over the instruction and said so in its report.
- High: do exactly what was asked, in small steps. One new script per answer: 17 scripts, 317 lines, the most code. Many short turns with the shortest reasoning per turn. Once it changed a bank name while only asked about an amount, and reported it.
- Max: build a tool first, then question everything. After the first answer it wrote a generic edit driver and from then on wrote only JSON per answer. It tested an uncertain function with a small script before using it. It was the only run that left an unknown name as “to confirm” instead of filling it in.
Their feedback on the skill differed too. Low listed where it got stuck, high listed practical annoyances, and max found contradictions between commands and docs: two check commands reporting the same result as pass and fail, and development notes leaking into the user-facing help.
Claude shows the same pattern
Sonnet on medium and high scored the same; high took 1.5× the time (5.7 → 8.5 min) and 1.5× the cost (USD 1.31 → USD 2.02). In an earlier benchmark, DeepSeek on medium opened the skill’s source code whenever the docs felt thin and wrote extra scripts, ending 10 minutes slower than low with the same accuracy.
Takeaways
- When procedure and tools decide the numbers, effort does not raise quality.
- Effort changes volume, not speed: max wrote about 2× the tokens of low at the same tok/s.
- Effort changes style: low batches, high follows literally, max builds tools and questions the docs.
- Sub-agents inherit effort, so the cost multiplies.
- Run on low. Use max as a reviewer of the procedure and tools, not as the producer.