The Real Cost of AI Efficiency Showed Up Elsewhere
This week’s most useful lesson was that AI efficiency is a system property. Memory reductions, fast kernels, and cheap decisions looked impressive in isolation, but storage, routing, context, permissions, and human review often decided the actual result. The strongest progress came when the task was narrowed and the measurement stayed close to the work.
Memory Savings Moved the Problem
DeepSeek cut cache memory per context token from 389,000 bytes to 890 bytes across four generations. At one million context tokens, that means 890 MB instead of 389 GB. It achieved this through shared caches, lower precision, sparse selection, and a design that reads selected positions rather than everything.
That is a major engineering result, but not a universal win. DeepSeek led Claude Opus 5 on Terminal Bench 2.1 by one and a half points, led by one-fifth of a point on a software engineering benchmark, then finished 20 points behind on Terminal Bench 4.0. Artificial Analysis measured about 200 tokens per second, but the model produced nearly twice the median output length.
Edge Zero made the same tradeoff more visible. It reported running a 35 billion parameter model with 2.9 GB of peak memory allocated by MLX, while the checkpoint totaled 19 1/2 GB. Most expert weights remained on the SSD or in the operating system’s page cache. The system loaded 283 megabytes of weights per token and reported up to 59% higher decode throughput.
The catch was performance under memory pressure. On an 8 GB M1, it produced about 0.7 tokens per second, compared with a project target of 13 tokens per second. Low allocator memory did not mean low total memory demand.
Spark made the same point from another direction. Its configured context was just over one million tokens, but the estimated KV cache was roughly 39 GB, compared with about 155 GB if all layers kept full history. Smaller model weights do not automatically make long conversations cheap.
The GPU database test was the cleanest surprise. The GPU kernel was about 3 times faster than the CPU kernel, yet the full cold pipeline took about 568 milliseconds on the GPU and 490 milliseconds on the CPU. The GPU kernel itself accounted for only 14 milliseconds. Moving data from storage to memory and then across PCIe mattered more than the compute benchmark.
Agents Are Becoming Systems, Not Just Chat Windows
Neelhorse improved the behavior of 4 billion and 9 billion parameter Qwen agents and in the official configurations reviewed by the source, the language model dimensions stayed the same. The reported 4 billion model average moved from 58.94 to 64.87, nearly reaching the original 9 billion model’s 65.6.
The important improvement was operational. In one repository task, the agent read the README, applied the local rule, preserved the command line interface, ran the command, and inspected both output files. That is different from merely generating plausible code.
The evidence is still narrow. These were the Neelhorse team’s numbers, not independent replications. Workbuddy used three runs, while some other benchmarks used only one. The paper covered one training pass and did not show whether repeating the process would keep improving results.
Jev represents a different kind of progress. It does not write answers. It chooses among options defined by the application, making it useful for routing, tool selection, and other bounded decisions. Responses often arrive in roughly 100 to 300 milliseconds. In one skill selection example, the wrong skill rate fell from 17% to 7.3%.
That speed comes from restricting the problem. Jev’s “zero hallucinations” claim refers to schema matching, not factual correctness. An allowed option can still be wrong, and ordinary code should remain responsible for fixed rules such as payment limits or account permissions.
Coding tools are also converging around persistent project context. Cursor Projects, Claude Code Projects, shared workspaces, subagents, external events, and AGENTS.md all move away from one isolated chat at a time. The likely benefit is continuity. The new risk is that more tools, permissions, and background actions create more ways for a mistake to reach the host system.
The Claims Are Narrower Than The Headlines
The best results this week came with clear boundaries. Jev’s vendor evaluation covered four workflows and reported 67.8% aggregate agreement, while an outside test found it caught six of seven intended defects compared with seven of seven for the comparison model. DeepSeek’s open kernel supported two Blackwell targets, did not support top K settings above 4,000, and received fixes for four reported bugs.
The same caution applies to computer use and media generation. Personal workflow reports described agents editing video, handling email, and producing creative work, but those were not controlled evaluations. Editable After Effects projects are more useful than flat exports because timing, text, sound, and objects can still be changed. They do not prove that the finished media works with an audience.
Even local coding demos need this discipline. One walkthrough repaired a CSV implementation until all seven tests passed, but it did not establish that the local model had generated the implementation. A clean test result can validate a replacement without proving how much autonomy produced it.
Frontier labs offered another example of a narrower response. Anthropic and OpenAI agreed to give outside evaluators access comparable to internal risk teams. That adds visibility, but it does not slow training. The concrete commitment was a camera, not a brake.
Counter-Thesis and Risk Watch
Alvin Grlin rejects the idea of an AI arms race with one finish line or winner. He expects smaller open models to spread high-quality intelligence across laptops and phones. In another account, Alvin argued for smaller specialized models and questioned whether the largest systems and data center investments are necessary.
Bull Theory on X reported that Michael Burry called OpenAI and Anthropic’s safety warnings “fake hype” intended to conceal slowing growth ahead of their IPOs. A reply said Ed O’Dowd appeared to agree that AI fear could encourage Congress to exclude smaller companies and cheaper Chinese LLMs.
These are attributed claims. The supplied material does not provide supporting evidence for the allegations.
Looking Ahead
Can independent tests reproduce the reported gains while measuring total memory, storage traffic, latency, permissions, and human intervention?
That is where this week’s numbers become useful. The winning benchmark was often the one that measured the whole system.