A specialized 9B model achieved 53.6% task success repairing agents. A prompted 397B model from the same lab and model family reached 45.8%.
This does not show that smaller models generally outperform larger ones. It shows that specialization and executable evaluation can beat scale on a specific harness repair task.
Reward the outcome, not the plausible patch
The evaluation rewarded a patch only when rerunning the tasks improved average scores. That matters because plausible changes frequently caused regressions. One patch reduced a batch from seven successes to zero.
A change can look reasonable and come with a convincing explanation while making the system worse. Appearance and reasoning quality are not enough. The target workflow has to run better under measurement.
The editor could add instructions, insert state-dependent hints, block or rewrite actions before execution, and provide recovery guidance after failures. Removing the pre-action guard reduced performance by 3.9 points. Removing post-failure recovery feedback cost 3.3 points. These interventions mattered more than generic notes or hints.
Across 63 model and benchmark pairs, 56 improved, four were flat, and three declined slightly. The average gain was 7.06 points.
The benchmark boundary belongs beside those results. The evaluation covered web shopping, a text household simulator, and a database. It did not include a coding agent or a production-like stack. The evidence supports workflow-specific evaluation, not universal transfer to every agent system.
Capacity does not guarantee usable context
Advertised context capacity tells us how many tokens a model can accept. It does not tell us how reliably the model will use them.
In cited tests, 11 of 13 models fell below half their short-context baseline at 32,000 tokens in the no lima sack test. Longer inputs were also associated with more retrieval errors, greater susceptibility to planted wrong answers, and premature termination.
Fitting information into a context window is different from reasoning reliably over it. Compaction, structured notes, RAG, and sub agents have been proposed as ways to keep context small and curated.
Recursive language models offer another approach. They reportedly allowed GPT5 mini to beat plain GPT5 by more than 30 points on olong at 132,000 tokens. But a depth-two reproduction turned one 3.5-second task into a 344-second task and produced worse answers. Recursion may help orchestration, but more processing can also bring dramatically higher latency without better output.
Routing gains must survive the handoff
NVIDIA’s KV cache translation work attempts to avoid repeating the full prefill when routing a request between compatible models.
In a Qwen 3 14B to 32B test at 32,000 tokens, translation took 278 milliseconds, compared with 6,975 milliseconds for repeated prefill. The translated cache retained 97.6% of the receiving model’s average accuracy.
The result points toward cheaper routed fleets, but only under strict constraints. The method currently works within model families, requires compatible KV geometry and dense attention, and needs directional mappers of roughly 1 to 3.44 billion parameters. Accuracy retention also varied sharply. Two Ministral handoffs retained only 44% and 42%.
This is not transferable memory between arbitrary models. Its value depends on the specific handoff, architecture, latency target, and acceptable accuracy loss.
Never treat more capacity or a plausible change as an improvement until the real workflow runs better under the constraints that matter.
Model specifications tell you what a system can hold or attempt. Executed results tell you whether it is dependable, fast enough, and worth paying for.