Unreal Agent cut token costs by nearly 40% on Terminal Bench while matching OpenAI Codex’s 57.9% pass rate. It reached that result without changing the model or tuning weights. Instead, Unreal Labs altered the harness to stop burning tokens on polling.
Unreal Labs reported Unreal Agent at $1,428 and Codex, the leaderboard baseline, at $2,350. Both systems recorded a 57.9% pass rate using GPT-6 Astra at X High with Codex and Pi. That works out to a calculated 39.2% cost reduction, which Unreal Labs rounded up to a 40% headline.
Stopping the Polling Loop Cut Token Costs by 39.2%
In typical agent workflows, the model repeatedly queries running operations, burning input and output tokens while waiting for tests or commands to complete. Unreal Agent changes this flow by running long operations in the background while allowing independent work, like reading files, to proceed in parallel. The harness logs execution state and calls the model only when finished output is ready.
These savings are self-reported by Unreal Labs without independent reproduction, and matching pass rates on one suite does not prove equal reliability elsewhere. Across other published Codex comparisons, the stated calculated reductions drop to 28.2% on Sweet Atlas, 16.3% on Deep Sweet, and up to 25.7% on ALECL.
Published data also reveals reporting discrepancies. For Sweet Atlas, the Unreal Labs article lists Codex at $1,303, while its linked Harbor overview displays $1,356.18. That Codex overview lists 372 finished trials, 354 completed, 18 errors, and an average reward of 0.60, while Unreal Agent shows 372 finished trials, 356 completed, 16 errors, an average reward of 0.63, and a 90% cash hit rate. Harness efficiency is real, but do not expect an automatic 40% discount across every task.
Unit Tests Pass, but the Button Still Breaks the Layout
Cutting token waste addresses only half of the execution problem. An agent also needs to know if the code it produces actually works. Passing unit tests and linters is necessary, but a new interface button can pass every test and still break page layout.
Normally, an engineer has to open a browser, click elements, check logs, and prompt the agent again to fix visual defects. Anthropic is moving those manual checks inside Claude Code with a built-in verify skill. On its first run, the skill executes the app, checks changes directly in the interface, and saves those verified steps as a reusable project skill.
In a recent demonstration, an operator asked Claude to add a Like button to an existing web page. The agent started the development server, opened the page, clicked the button, and took a screenshot. It then ran a performance trace using Google Chrome’s DevTools MCP.
That trace identified layout shift, where page elements jump while content loads. Claude fixed the layout shift and re-ran checks until they passed. Codifying manual routines into reusable project skills allows coding agents to verify their own work autonomously with fewer prompt rounds.
Mobile Automation Forces a Trade-off Between Speed and Verification
Moving agents from desktop web browsers to mobile screens introduces different engineering hurdles. Google published an open-source repository called Artemis under the Apache 2.0 license to turn natural language into Android actions. It provides a native Model Context Protocol server connecting to Claude Code, Cursor, Windsurf, Codex, and Antigravity, driving a USB-connected phone or emulator using ADB, Scrcpy, FFmpeg, and Python through UV.
Many Android applications built in Jetpack Compose or Flutter feature thin or empty accessibility trees. To locate interactive targets, Artemis combines remaining accessibility hierarchies with OCR and visual models. The software exposes two operational modes depending on how much verification an engineer needs:
- Flash mode: Runs a reactive loop with a step time of roughly 3 to 5 seconds, skipping graph orchestration, pre-execution checks, checkpoint verification, and an ADB shell.
- Pro mode: Requires roughly 15 to 40 seconds per step, relying on a multi-agent planning and verification graph that checks targets against the live UI tree and pixels before dispatching actions.
Artemis reported a 99%+ completion rate on the Android World benchmark, which covers 116 programmatic tasks across 20 real-world applications. By comparison, the Android World paper authors stated that their own best agent completed “30.6% 6%” of the tasks.
These results require significant caveats. The 99%+ completion figure is self-reported on a Google sheet without independent reproduction, and testing on a clean Pixel 6 running Android 13 API 33 does not show how an agent handles custom OEM skins, live hardware, or personal accounts. Granting an autonomous agent accessibility access and an ADB shell also introduces serious security risks.
Probing Internal Activations Catches Bad Reasoning at a Compute Cost
While external harnesses monitor application interfaces, researchers are probing internal activations to catch flawed logic before generation finishes. A study that first appeared on February 1st, 2026, with a May 11th revision, examines two signals: value, which estimates the chance of eventual success, and temporal difference error, which tracks changes in that estimate. The authors call these groups value neurons and dopamine neurons.
Using a 7 billion parameter Qwen model fine-tuned for reasoning, the researchers recorded an original MATH 500 accuracy of 75.2%. Zeroing the selected 1% of coordinates in layer three collapsed accuracy to 13.6%, whereas zeroing the same proportion at random in that layer left accuracy at 73.4%. That collapse confirms those specific coordinates carry critical reasoning signals.
The researchers then trained a probe to evaluate candidate reasoning steps at paragraph breaks. For the search test, the model generates four candidate continuations at a paragraph boundary. Greedy decoding and random selection both scored 72.2% accuracy, while an alternative reward-based method reached 75%.
The dopamine probe reached 77.8%, representing a gain of 5.6 percentage points on the MATH 500 validation subset averaged over three random seeds. The method retains steps whose estimated change is larger than 0.3 in magnitude. Before generating an answer, a sparse probe also predicted confidence with an average AUC of 0.67, outperforming self-reported model confidence at 0.52.
The core constraint is compute. Generating four candidate branches at every step requires four times the generation work, burning significant compute. Additionally, the study normalized activations across the entire response before averaging by paragraph, a lookahead step that live, real-time scoring systems cannot perform.
Harness efficiency is real, but until independent testing reproduces these cost cuts across varied tasks, treat self-reported benchmark discounts with skepticism.