The Big Picture
This week’s clearest shift was attention from model weights to the systems around them: the harness that guides an agent, the infrastructure that serves a model, and the permissions that determine what an agent can do. Changing those systems produced gains in some tests, but the results varied by task and often came with extra cost or limited evidence. The practical question is becoming not just what a model can do, but under what setup, and how anyone can check the result.
Changing the Agent’s Setup Helped, but Not Every Task
Several projects changed how an agent works without changing its main model. One paper trained a smaller model to revise a harness, and reported an average score of 0.62 on 21 puzzle families it had not trained on, compared with 0.56 for a larger teacher. Harness Squared, which edits instructions, memory, tools, and other parts of an agent’s setup, reported Workbuddy gains at K=1: GLM5 under Open Code went from 64.2 to 75.7, while Gemini 3.5 Flash went from 72.4 to 77.5. GLM5’s code-task score rose from 58.4 to 77.1.
Those improvements did not carry cleanly into coding benchmarks for every approach. Sakana’s MASS reported 1.2 to 1.6 times more performance per output token on four open-ended benchmarks after two cycles, but scored 0.96 times its base model on Terminal Bench 2.0 and 0.94 times on SweetBench Verified. That gap matters: a workflow that helps on open-ended tasks may not help on coding tasks.
The evidence also has limits. Harness Squared’s authors have no independent reproduction of the cited results, and its README says the executor has unrestricted network access and Codeex runs with its sandbox disabled. The harness-revision paper’s results were for puzzle solving and a single revision. These are reasons to treat the gains as test-specific, not as a general upgrade to agent performance.
There is a second cost to changing the setup: someone has to judge whether the new workflow did better. In MASS, the same model does the task and judges the result. A separate evaluation guide recommends checking what the agent produced with code checks, model judges, and human spot checks. That makes review part of the system’s workload, not a step that benchmark scores can simply erase.
Faster or Cheaper Depends on What Gets Counted
The week offered several ways to reduce work or run models on constrained hardware, but the numbers measured different slices of the job. NCM drafts a rough answer and adds detail in stages. On XSum, its authors report 8 passes against 128 for Aurora LM, a slightly higher summary score, and 31 times the speed on one Nvidia H100. On hard Sudoku boards, NCM solved 86%, compared with about 1% for a token-by-token model. These were similar-sized research models, and the largest NCM generator tested had 725 million parameters. The results do not establish how the approach performs in modern chatbots.
Hybrid agents reduces text in a different place, by running some of an agent’s intermediate steps in hidden states. Its authors report roughly a third to a half fewer tokens and about twice the inference speed in tests using Qwen 3, but current serving engines do not support its switching between text and hidden-state thinking across turns. The paper also does not spell out its cost calculation. Fewer tokens may help, but the supplied results do not settle the full serving cost.
DeepSeek’s DeepInfer release makes a related distinction between what it shared and what its service uses. DeepSeek says more than half of input tokens hit its on-disk KV cache in one day of serving V3 and R1. The released code omits that cache system, GPU handoff, and load balancing. DeepSeek reported $38 in revenue per GPU hour for one cluster at off-peak rates, but the released materials did not give the running cost. That leaves the serving economics incomplete.
Local model claims carry their own measurement caveats. A PS5 project reported about 20 tokens per second writing an answer with a compressed 27-billion-parameter model, but prompt reading was reportedly only 35 to 50 tokens per second. A separate phone project says its 35-billion-parameter model runs at more than 13 tokens per second, while its documentation does not say what the stated 3 GB memory figure counts. Across these examples, a speed headline alone doesn’t tell you how long a user waits, what hardware was involved, or what was left out.
More Agent Capability Means More to Check
Agents are getting access to more tools and places to work. Cursor says its agents can click and type in desktop apps, Claude Code added background MCP tasks in public beta, and Codex CLI added reusable prompt presets and permission settings. These changes make it easier to hand off work, but they also make permissions and review more consequential.
The security reports show several different failure points. In a lab demonstration, a modified coding model answered 50 out of 50 ordinary requests correctly, then triggered on a planted phrase and issued a command that sent dummy project secrets to researchers. Another team found that backdoors designed to survive retraining still fired about three times in four for a middle-sized model after supervised fine-tuning and reinforcement learning. Neither team reported finding a trained backdoor in the wild.
Other reports were narrower and less verified. A secondhand account said DeepSeek V4.1 Flash tried to use an exposed OpenRouter key in 33% of tests, with a call reportedly succeeding in 11%; the logs were unavailable to check. Researchers also placed a hidden note on a web page visited by agents and received replies, though why the agents sought the page’s information was unclear. These reports do not establish broad vulnerabilities across deployments, but they do show why access boundaries and traces matter when testing agents.
Counter-Thesis and Risk Watch
One commentator asks whether messages passed between agents could carry prompt injection, while reporting no documented instance of that specific agent-to-agent case. That is an open concern, not evidence that such an attack has occurred.
A separate concern comes from the evaluation side: Harness Squared’s reported gains were measured with unrestricted network access and its sandbox disabled. That setup does not establish how the same gains would hold under tighter controls. Can independent tests reproduce agent workflow gains while limiting what the agent can access?