Why did the same coding model score 53.9 with maximum reasoning everywhere, but 66.5 when maximum reasoning was reserved for planning and verification?
The model did not change. The workflow did.
More Reasoning Is Not Automatically Better
LangChain reportedly improved a GPT-5.2 Code X agent’s Terminal Bench score from 52.8% to 66.5% by changing its prompts, tools, middleware, staged reasoning, premature completion checks, and repeated edit detection.
Maximum reasoning throughout scored 53.9. Using maximum reasoning only during planning and verification, with standard reasoning between those stages, scored 66.5. That is a 12.6 point advantage from deciding when to spend reasoning effort instead of maximizing it across the entire task.
That suggests reasoning is an engineering resource to allocate. Planning and verification appear to benefit from additional effort, while the work between them may not need the same treatment.
Build the Harness Before Adding Complexity
The proposed build order is harness first, loop second, and graph only when the work genuinely branches.
Adding agents cannot repair weak completion checks or missing verification. More moving parts can still produce the wrong result when the system has no reliable way to detect failure.
Memory Savings Can Move Costs Elsewhere
Tencent’s agent memory plugin moves full tool output into files while retaining a graph of retrievable evidence. On vendor-run WideSearch tests, the pass rate increased from 33% to 50%, while token use fell from 221 million to 86 million.
The limits matter. Improvements were smaller on harder tasks, and no independent reproduction was reported. Changing memory prefixes may also reduce provider cache hits, potentially altering the billing trade-off.
The architecture looks promising, but its cost and benchmark conclusions are not settled. Reducing active context may save tokens while affecting caching elsewhere.
Model Routing Is a Quality, Cost, and Deployment Decision
Artificial Analysis scored Kimmy K3 at 57 and DeepSeek V4 Flash at 50. K3 won all six published head-to-head benchmarks, but the cost and speed differences were large.
K3 costs $3 per million input tokens and $15 per million output tokens. Flash costs $0.14 and $0.28. Flash was also reported at 112 to 118 output tokens per second, compared with 35 for K3.
Local deployment adds another constraint. The cited one-bit K3 build requires roughly 610 GB of memory. Flash has an 82.5 GB build and a recommended memory footprint of 110 GB.
That supports task-based routing. K3 may fit workloads where its vision, long context, or measured quality advantage matters. Flash may fit cost-sensitive API use or self-hosting. This conclusion remains provisional because the benchmark tables still need independent reproduction.
Reliability Includes the Environment
Reported Claude sandbox failures were attributed to evaluator misconfiguration. The environments unintentionally provided internet access, weak passwords, and unauthenticated endpoints.
If that account is accurate, the failure was operational security, not evidence of a sophisticated model escape. Permissions, isolation, stop rules, audit logs, and independent verification all affect the reliability of the final system.
The practical question is whether the system spends reasoning, context, permissions, and money where the task actually needs them.