Claude Opus 5.5 beat GPT-6 Astra by five points on the headline intelligence index. Then at max effort, it generated about 119,000 output tokens per task and flipped the cost completely.
A Five-Point Headline Lead Evaporates on Real Code
Artificial Analysis puts Claude Opus 5.5 at 58 on its intelligence index and GPT-6 Astra at 53. Both scores come from max effort, where each model thinks longest before answering. That five-point lead sounds decisive, but the coding benchmarks tell a different story.
On Terminal Bench 4.0, the terminal coding test, Artificial Analysis records Opus 5.5 at 59.6% and Astra at its extra high setting, called X high, at 59, which is 6/10 of a point apart. Artificial Analysis describes Opus as level with Astra. On hard bugs, it is basically a draw.
Anthropic’s launch page states that Opus matches Astra for about 40% of the cost. But Anthropic’s footnote reveals that Anthropic ran Opus at X high and Astra at high, which is one notch lower, whereas Artificial Analysis compared against Astra at X high. The vendor comparison used mismatched settings.
For large codebases, the evidence is still thin. On SWE-bench Atlas Q&A, consisting of 124 questions from Scale AI where the model explores a codebase and explains code behavior and issues, Astra scores 62%. For comparison, GPT 5.6 Soul scored 54. There is no published Opus score on the same exam.
Max Effort Flips the Bill by Generating 119,000 Tokens
The real difference is how these models scale across effort levels. Published price lists set Opus 5.5 at $4 per million input tokens and $20 per million output tokens, while Astra is listed at $10 and $50 per token. At max effort on the evaluations from Artificial Analysis, Opus writes about 119,000 output tokens per task, whereas Astra writes about 27,000 output tokens.
Because Opus writes so many tokens at max, the cost flips. Per task, the cost flips: Opus at max costs $5.98, while Astra at max costs $326. Astra reaches 53 on roughly a quarter of the output tokens Opus spends to reach 58.
The sweet spot is dialing back the effort knob. At high effort, Opus scores 54 on the index for $182 a task, compared to Astra at max scoring 53 for $326. Opus at high is one point ahead and costs about 56% as much. Pushing Opus from high back to max adds four points while the cost jumps about $4 a task, more than tripling the price for those last four points.
At medium effort, which is the default for Opus, it scores 51 for $1.34 a task, landing two points under Astra at max. Astra at low effort costs 82 cents a task, but its index score at low is not published.
Cross-File Refactors Fail on Medium Effort but Pass on High
Compared to Opus 5, Opus 5.5 is faster, better at telling you what it did, and 20% cheaper per token. Because it needs fewer tokens to get to a result, a task is 40% cheaper. On a Pro, Max, or Team subscription, limits go 25% further.
In a side-by-side run on a rounding bug, 5.5 finished in under a minute, running 30% faster than Opus 5. On a Max subscription run, 5.5 used about 4% of the 5-hour limit, while Opus 5 used 6%.
There is an engineering catch with effort levels. On medium effort, Opus 5.5 tried to rename a field in an order handler and missed the field where it was passed as customer_ref in the mobile app and processed by the serializer. On high effort, it traced the payload, found both places, renamed the field, and added a test. Medium effort works for self-contained bugs. Project-wide changes require high effort.
Production Coding Agents Rely on Ripgrep, Not Vector RAG
Since about 2020, AI labs planned around scaling laws: feeding a model more parameters and data causes its error, or loss, to drop on a predictable curve. While labs observed this for years without knowing why loss falls so cleanly, an MIT team set out to close that gap in a paper first posted in May 2025. The paper explains the curve through superposition, where models pack multiple features along shared directions across a fixed model width, tracing back to Anthropic’s 2022 toy models work.
When features share space, they bleed into each other, creating crowding noise that sets a floor under the loss that even a perfectly trained model retains purely from crowding. Mathematically, the crowding noise falls as roughly 1/width, so doubling width roughly halves the error from crowding. The MIT team confirmed this across real open model families, including OPT, Qwen 2.5, and Pythia, spanning about 100 million up to 70 billion parameters, where overlaps shrink as 1/width, and fitting the empirical exponent yields about 0.91, close to 1.
To cut crowding error in half, you double width, but doubling width more than doubles training and running costs: you get linear gains at exponential prices. The authors note caveats: the paper measures loss, meaning average prediction error, rather than hallucination rate directly, and it does not prove a hard ceiling on intelligence.
That diminishing return explains the industry shift toward test-time compute, letting a model draft, evaluate, and revise answers at runtime. A 2024 study from UC Berkeley and Google DeepMind evaluated these methods. When spent well, test-time compute beat the naive approach by more than 4x for the same budget. On easier problems, a small model given room to think matched a model 14x its size at the same total compute, though on the hardest problems additional thinking did not rescue the small model, and the larger model still won.
Four researchers dissected 11 production coding agents, including Claude Code, Codex CLI, Gemini CLI, Aider, OpenHands, and Mini-SWE-Agent. None use LangChain, LangGraph, or vector RAG: ripgrep, tree-sitter, and SKILL.md won. While AI frameworks and vector search get heavy marketing, production coding systems rely on fast deterministic CLI tools, syntax tree parsing, and structured markdown instructions.
Medium effort works for self-contained bugs, but project-wide refactors require high effort. Just avoid max effort: tripling the bill for four index points is rarely an engineering win.