DeepSeek V4 Pro scored 87.9 and 79 on Terminal Bench 2.1. Both results can describe the same model because the higher score included DeepSeek’s harness, minimal mode, and maximum reasoning effort.
The 87.9 result is not a measurement of model weights in isolation. It measures a model, harness, mode, and reasoning configuration working together.
The harness is part of the score
DeepSeek released its harness under the MIT license alongside V4 Pro. It treats models, tools, skills, sessions, sandboxes, files, and interfaces as replaceable plugins. An append-only event stream records runs and supports replay, search, resume, and branching.
These capabilities could improve the reproducibility of agent evaluations. The harness is explicitly a developer preview, however, and reproducibility does not guarantee comparability. It adds another configurable layer and another surface vendors can optimize.
The model weights did not win independently. The configured system produced the result.
Cheaper depends on the workload
Under Artificial Analysis’s 70/20/10 workload mix, one calculation puts DeepSeek at about 35 cents per million blended tokens off-peak and 69 cents at peak. Gemini 3.7 Flash comes in at about 58 cents.
DeepSeek is cheaper per completed task in that comparison, but other measurements change the ranking.
Gemini has an Artificial Analysis index score of 56, compared with 53 for DeepSeek V4 Pro. In the cited third-party measurement, Gemini produced about 340 output tokens per second while DeepSeek produced about 83.
Latency points the other way. Gemini took nearly 10 seconds to produce its first token. DeepSeek took under two seconds.
Gemini’s higher output throughput could fit an interactive product where finishing quickly matters. DeepSeek’s lower calculated task cost could fit scheduled, cache-heavy work. Neither choice follows from a single score or price.
Token price is not system cost
Pricing schedules and workload assumptions can reverse a comparison. DeepSeek’s August 16 schedule sharply raises peak pricing and reduces the value of cached input. Gemini 3.7 Flash launched on August 13 with promotional pricing of 75 cents per million input tokens and $3.75 per million output tokens. Google’s published schedule doubles that price on January 1, 2027.
Cache behavior, peak timing, input and output mix, reasoning effort, and throughput all affect total cost.
Agentic systems amplify the problem. Anthropic estimates that multi-agent systems consume roughly 15 times the tokens of simple chat. Microsoft and Stanford’s Digital Economy Lab put agentic tasks near 1,000 times a chat message.
At those multipliers, small token-price differences can become material. So can retries, reasoning configuration, and failed runs. Cheap tokens do not guarantee a cheap system.
Security determines whether the workflow is usable
Grockbot provides a persistent cloud computer with a browser, filesystem, terminal, shared logins, and reusable routines. Its bots share one computer, filesystem, browser sessions, and set of credentials.
XAI’s documentation says separate bots must not be treated as a security boundary. An audit view is still forthcoming.
Access design is therefore part of workflow usability. A cautious operating posture is to start with read-only access, connect only required services, and require approval before sending, purchasing, or deleting.
Compare completed workflows: hold the task constant, include every configuration and operating cost, and require reliable completion within an acceptable security boundary.
The useful question is not which model won. It is which combination of model, harness, configuration, workload, and security boundary completes the task reliably at an acceptable total cost.