Jev made decisions in a median 274 milliseconds for seven cents per thousand in an independent banking test. For a tool that chooses where messages go, those numbers are appealing. But the result I would look at first is what happened when no choice fit, or the message did not settle the answer.
On those items, Jev’s confidence fell to 0.5 or below only 49.7% of the time. Speed helps once a route is clear. The harder case is a message that has no clear route at all.
A Fast Choice Can Still Be the Wrong Route
Jev takes a message and a list of choices, then returns a probability for each choice and a confidence score. It does not write its answer in prose. That makes confidence part of the result: when the message does not support a clear choice, the score helps determine whether the selected route should be trusted.
In Nibsard’s independent banking test, Jev got 76.3% right on 200 items. On items without a clear answer, seven of eight LLMs put confidence at or below 0.5 between 97.3 and 100% of the time. Jev did so 49.7% of the time.
That gap matters for the job Jev is doing. A confident wrong choice can send a message down the selected route instead of to the branch that asks for help. I would want to understand that behavior before treating the speed and price as the whole result.
Choice order also affected its answers. When the choices were shuffled, Jev changed its answer on 13% of items. That is a separate stability problem for a tool whose output is a choice from a supplied list.
The speed comparison is still strong within this test. GPTOS 120B on Cerebras took 331 milliseconds and cost 32 cents per thousand decisions. Jev took 274 milliseconds and cost seven cents. Jev also has a documented limit of 255 choices; at 256, the test received an HTTP 400 error.
There are limits to what the banking results settle. The report gave no confidence intervals or significance testing. A separate pilot used a different banking setup and reported 87% accuracy and a 0.054 calibration error across 300 examples. We do not know why the two results differ.
Strata’s Speed Came From Where It Put the Model
Strata reported 44.8 tokens per second at 128K context for Qwen-3 8B Flash Next in IQ3_XXS format on an RTX 5070. Its table reported 65.6 at 4K context. The placement of the model’s pieces helps explain how Strata reported those speeds.
The core has 125B parameters. An N-gram table and a draft layer bring the total to 180B, while about 6 billion core parameters are active for each token. Strata keeps selected experts in a roughly 6 GB GPU cache, so the whole model does not have to fit on the GPU at once.
If a needed expert is missing from that cache, the CPU handles it in system RAM while the GPU works on its part of the same token. The N-gram table stays on the SSD. A draft layer proposes tokens for the full model to check together.
Those are Strata’s own speed measurements. The repo docs do not specify the 15 tokens per second baseline behind the circulated “3x” claim. The speed table does not measure answer quality, and the code’s license field is empty. The placement is interesting, but the reported speed answers a narrower question than overall model performance.
Coding Tool Numbers Need Their Labels Too
Codex CLI 0.158.0 added copy on select, right click paste, transparent backgrounds for image generation, and OAuth client secrets for MCP. Elevated command approval is now on by default. OpenAI also shipped fixes for the reported Heapjack and Overpatch sandbox escape bugs in Codex CLI and Desktop.
Users reported a CMD+C problem in remote sessions, slow TUI scrolling, and displeasure with the refreshed interface. Those reports sit alongside the shipped changes, but they describe user experience rather than a shared performance test.
The other coding tool claims have different sources and scopes. Users and Anthropic staff described Claude Sonnet 5.5 in Claude Code workflows as about 30% faster and cheaper than Sonnet 5. A SpaceXAI engineer shared a walkthrough of one engineer producing 2,500 PRs in a month with Cursor agents. Another post compared how long a $100 Claude plan and a $200 Codex plan lasted for similar work. These are product and user reports, not results from a common benchmark.
For Jev, the unresolved question is more specific: when a real message has no valid route, how often does its confidence fall below 0.5?