Prime Intellect reported moving Opus 5 from 30% to 95% on ARC-AGI 3 in 12 days without retraining it. The change was a recursive harness, not a new model.
That unusually large gain does not prove recursion improves every task. It does show why an agent’s practical capability cannot be judged from its model name alone. The operating loop around the model matters.
The operating loop can matter more than the model upgrade
MIT reported a similar result: GPT-5 mini driving Python beat plain GPT-5 by 34 points on a 132,000-token benchmark, at roughly the same cost per query.
The conclusion is not that smaller models are always better. The comparison shows what changes when one system can select context, execute a tool, inspect the result, and try again, while another receives a single large prompt.
Model-only comparisons are incomplete when the surrounding systems work differently.
Useful context is maintained state, not an ever-growing prompt
Long-running agents need the right state, not every previous token.
OpenAI’s internal project reportedly exceeded one million lines and roughly 1,500 pull requests, with no human-typed code. Some Codex runs exceeded six hours. Those runs used maintained state and structured handoffs instead of carrying their entire history forward on every turn.
The emerging design separates four concerns:
- Durable instructions
- Current decisions
- Discoverable history
- A bounded stopping condition
An agent needs enough information to understand its current task, recover the decisions constraining it, locate relevant history, and know when to stop. State management is therefore part of system capability, not merely a prompt optimization.
Harness cost appears in the completed task
A more capable operating loop can cost much more than the model price suggests.
One speaker estimates that agent or multi-agent systems can cost four to 15 times more than chat. Harness prompts can add 8,000 to 30,000 tokens, while tools create repeated model calls. These figures come from one speaker, and some associated comparisons omit units, so they should not be treated as a general cost model.
The narrower conclusion is still useful: cost per model call is not cost per completed agent task.
The full cost depends on how much context the harness adds, how often the model runs, how many tools it invokes, and how many attempts the workflow needs.
Permission boundaries are part of capability
The NPM ecosystem shows why tool access cannot be separated from security design. In August 2026, the Keev campaign reportedly affected more than 400 package names and over 2,000 poisoned versions. It also planted persistence in configuration files read by coding agents.
Those configuration files belong to the agent’s trusted context. If an attacker can place instructions there, the operating environment itself can become a persistence mechanism.
Package selection adds another risk. JFrog found that 54 of 55 advisories from one account were fabricated. Separate studies reported invented package-name rates near 20% across 16 models, and between 4% and 6% across five frontier models this year.
These results do not establish a universal hallucination rate. They do support one clear boundary: an agent recommending a package must not implicitly authorize that package’s code to execute.
Pinned lock files, disabled install scripts, delayed upgrades, and explicit reviewed approvals separate recommendation from execution. That separation is part of whether an agent can be trusted to operate.
Judge the loop, not the label
The broader harness argument still has limits. The largest capability gains are reported benchmark results, not broad evidence across tasks. Early multimodal tooling lacks benchmarks and recorded cases. Richer scaffolding can increase token consumption, tool calls, latency, failure surface, and security exposure.
Never evaluate or authorize an agent from its model name alone.
Inspect what context the system selects, what state it preserves, how often it calls models and tools, and which actions require reviewed permission.