All seven coding tests passed. But the local model did not produce the repaired implementation during the research.
That distinction matters. A final result that works is not proof that the model being evaluated can produce it.
Seven Passing Tests Did Not Prove the Model Could Finish
The task was to build CSV export for an expense tracker. The tests included awkward labels containing commas, quotes, and line breaks.
The first implementation produced a clear result:
- Passing tests: 4
- Failing tests: 3
- After the CSV escaping rules were repaired: 7 passing tests
The task was real, the test suite was concrete, and the standard was easy to understand. But the local model did not make the repair during the research.
The walkthrough also separated the agent from the model. The agent reads files, edits code, and runs commands. The model produces responses.
Both are required for a working coding system. So are the right permissions, enough context, acceptable latency, manageable cost, and enough visibility to understand a failure. Testing the model alone misses much of what determines whether the work gets finished.
Jev Moves Small Decisions Out of the Main Model
Jev is designed for a narrower part of this system. Instead of generating a long answer token by token, it receives text, a prompt, and predefined answers, then returns probabilities that software can use directly.
The proposed role is routing: let Jev make frequent, structured decisions while a slower model periodically reviews the strategy and rewrites its criteria or thresholds.
In TypeSafe’s official skill selection example:
- Wrong skill without Jev suggestions: 17%
- Wrong skill with Jev suggestions: 7.3%
The speaker estimated that this approach could remove about 10,000 skill description tokens from the main model’s context. That matters because those descriptions would no longer need to sit beside the task itself every time the agent decides what to do.
Jev also screened 150 code comments in 9.3 seconds for about one cent. An estimated exhaustive pass for supported code smells used 28 million input tokens and cost $1.19.
According to the speaker, an engineer at Sentry reported that Jev was more than five times cheaper and faster than the smaller domain models already used in a security pipeline, while maintaining higher accuracy.
These are reported examples, not proof that the same split will work for every workflow. They do show what can be moved away from the main model: small, repeated choices with predefined answers.
Local AI Still Has to Complete the Same Work
An 18 GB model download does not describe the complete memory requirement. Alama recommends at least 64,000 tokens of context for coding agents, while its documented default below 24 GB of graphics memory is much smaller. Longer context needs more memory.
Price comparisons have the same problem. Cursor Pro was listed at $20 a month and GitHub Copilot Pro at $10 a month, for $30 a month, or 360 across a year at those prices.
The local run was not measured for completion time or energy use, so the comparison remained incomplete. Download size, subscriptions, and token prices do not tell us how much work the system can finish or how much intervention it needs.
The Hermes Agent speaker described local AI as free, unlimited, and without guardrails, and called current local models “Opus 48 quality.” On the speaker’s Mac Studio, Hermes recommended Quen 38 Flash.
But the same speaker still preferred frontier cloud models for complex applications, advanced knowledge work, and difficult multi-step tasks. Those positions do not fully fit together, because the difficult work is exactly where that preference remained.
Task cost also depends on how the system runs. Claude says Fable 5.1 has the same token price as 5, while cache reads cost 75% less. Claude says that cuts typical work costs by around 25%, and long agentic task costs by up to around 45%.
On the third-party CursorBench coding benchmark, Fable 5.1 at medium effort produced a similar result to 5 at maximum effort for about one-fifth of the cost. That was one benchmark, but it shows why token price alone is incomplete. Model choice, reasoning effort, caching, and the work required to finish all affect task cost.
The Generation Worked, but the App Rejected It
A sponsored Higsfield walkthrough showed the same system problem from another angle. The first image generation succeeded, but the app rejected its status link because Higsfield returned a platform-domain URL while the app expected the API domain.
After the integration was fixed, the completed job was recovered without paying for another generation. Visibility into whether credentials loaded, the provider accepted the request, the job was still processing, and a file had been returned prevented a duplicate request.
The Soul 2 image cost 1 cent. It became the input for a downloadable 5-second Cance 2.0 video with generated sound, priced at $1.29.
A successful generation was not enough. The surrounding system also needed to understand the provider’s response and preserve the result. Before opening the app to other users, the speaker would add sign-in, spending limits, and durable output storage.
The practical test is whether the complete setup can finish the identical task, pass the identical tests, and require less intervention.