One RTX 3090 report reached 140 tokens a second for code generation. Its reported median on a dense 27 billion parameter model was about 36 tokens a second.
Those numbers came from different workloads and setups. The larger one is real as a reported best case, but it does not tell you what every request on that card will feel like. The same care applies to price, memory use, and results from agents working outside an inference test.
An RTX 3090 Speed Number Needs Its Setup
The June eBay sold median for an RTX 3090 was $946. Other figures were $1,050 in a used-price article, about $1,450 in September, and a $1,850 October minimum that one commentator thought was probably retail. Putting those figures side by side is useful only if you keep track of what each one measures.
Llama Perf has the same issue on the speed side. It collects reports from individual setups, not controlled benchmarks. It had 189 reports for the RTX 3090, seven for the Arc Pro B60, and three for the RTX 4060.
The 140-token RTX 3090 result was a code-generation best case using multi-token prediction: upcoming tokens are drafted so the full model can check them together. One commentator says chat will be slower. Even before comparing cards, the workload matters.
For the same dense 27 billion parameter model, Arc Pro B60 reports included 53.3, 15.3, 22.0, and 22.2 tokens a second. That spread comes from reports for the same card and model, so one posted rate cannot stand in for every setup.
Model choice changes the comparison too. The 8 GB RTX 4060 reported 6.0 tokens a second on that dense model with CPU offload, and 24.2 on a mixture-of-experts model. The latter uses only some of its parameters for each token. Those rates describe different jobs.
Keeping Experts in GPU Memory Changed One Test
A llama.cpp change kept frequently used experts in GPU memory. In one RTX 5090 test, speed went from 30.8 to 67.8 tokens a second, with an 89% cache hit rate. That is a large gain in that test, but it supplies no rate for another setup.
There is also a difference between serving one conversation and serving several together. A local model keeps its weights in memory for one conversation. A server can share one copy of those weights across a batch, while each conversation still needs its own KV-cache memory.
A 2023 VLLM measurement gives a concrete example. With a 13 billion parameter model on a 40 GB Nvidia A100, shared weights used about 65% of memory and the batch’s KV caches used about 30%. That is an older setup; one commentator says OpenAI’s own numbers are not public. Sharing weights also leaves open why someone might want their prompts to stay on their own machine.
Websites Still Decide What an Agent Can Do
An agent reaching a website presents a different measurement problem: recognizing the agent does not by itself decide what it may do. The site still has to grant access.
Visa, Bill, and Bill Trust announced an October 20 pilot in which a registered agent pays an approved supplier invoice through the supplier’s portal. The receiving system recognizes the agent. Cloudflare can verify a digital signature on agent traffic, but the website decides what access to allow.
Shopify’s October 28 expansion lets merchants publish facts for agents, such as return windows and shipping destinations. One commentator says Shopify and Walmart support agent access, while Amazon has chosen against it. They also say there is no shared rule set yet.
Using an ordinary website still leaves practical obstacles: a page may change, an account check may appear, or the agent may be unable to finish the task. Forever Security says it stopped a coding agent from installing malware after finding a malicious instruction file in a customer’s downloads folder. How the file arrived is unknown.
The Minecraft Agents Coordinated Without Thriving
YC Labs gave 50 agents the same model, GPT 6.1 Soul, and asked them to build a thriving civilization in Minecraft. In three hours, they produced rules, permits, a ration ledger, and about 5,900 messages. YC says the colony did not thrive.
Ada became a point of contact. Of the other 49 agents, 46 sent their first message to Ada, who was listed first in the directory; she received 616 messages. The directory’s sorting rule is undocumented, and no separately shuffled list was tested. Her position raises a question about the tool’s influence, but it cannot settle why she became the hub.
Some coordination did work. After fishing instructions were shared and a dozen agents took shifts, fishing became a dependable food source. The ledger tracked reported health and hunger while warning that its entries were not live.
A ration handoff exposed another gap between a tool response and what happened. Ada tossed a cooked chicken toward Esme, who picked it up and ate it. Because the give tool could report a pickup without identifying who picked up the item, the agents wrote a rule: put rations in named chests and count someone as served only after they confirm they have eaten.
A reported token rate has a similar need for context. It may describe one workload, one setup, or the total across parallel requests rather than the speed one person sees.