A roughly 104 GB model ran on a 32 GB Mac mini. In a small answer check, though, its median MMLU Pro question took about four minutes. A smaller model answered in 38 seconds.
The setup made the large model runnable within the Mac’s memory budget. The answer check showed what waiting for it could feel like, without giving us enough evidence to say which model was broadly better.
A 104 GB Model Ran From Storage
In the reported test, Qwen 3.8 Flash Next read parts of the model from a USB 4 drive. It has 125 billion parameters, but about 6 billion work on any given token. Its router can choose different experts for the next token, so the setup kept busy experts in memory and read others from storage in chunks of about 2.8 megabytes.
Slot Stream tried to start those reads early by running each layer’s router two layers ahead. Flash Next generated four to five tokens a second. At the same memory budget, with speculative decoding, OMLX reached 4.8 and Slot Stream reached 4.4 tokens a second.
The smaller 27B model stayed entirely in memory and generated 28.4 tokens a second. That is the practical tradeoff in this test: storage let the much larger model run on the Mac mini, while the model that fit in memory produced tokens much faster.
The Sampled Answers Took Much Longer
Speed alone cannot tell us whether the wait bought a better answer. Flash Next’s model card reports 91.7 on GPQA Diamond, compared with 89.2 for the 27B model. In the reported answer check, both models got the same six of nine MMLU Pro questions right and the same four of five GPQA Diamond questions right.
The wait was sometimes striking. One chemistry question took Flash Next about 76 minutes. It reached the right answer in its reasoning, then argued itself out of it. With web search available, it spent 20 minutes before making four searches and ran out of time without an answer.
That check covered nine questions out of 12,032 on MMLU Pro, plus five GPQA Diamond questions. It gives little basis for a broad quality comparison. The longer wait on these sampled answers is much clearer.
K3 Hosting Has a Different Memory Problem
The Mac test shows one way to run a large model with limited memory. The available figures for hosting Kimi’s K3 point to a much larger memory requirement. K3.1 Pro’s files are not out yet, so these sizes and speeds belong to K3, not K3.1 Pro.
K3 has 2.8 trillion parameters, with about 104 billion working on a token. Its router picks 16 of 896 experts, plus two shared experts. Because the next token can call for a different set, all 896 experts have to stay in fast memory.
Unsloth lists a lossless copy at 1,560 GB. Its smallest 1-bit copy is 594 GB and needs about 610 GB of combined RAM and graphics memory. Unsloth says that copy picks the same next token as the full model about 79% of the time; its effect on coding quality has not been measured. Moonshot also says the open weight release removes cyber-related data from both training stages, so it is not quite the model behind the API.
The reported speeds need just as much care. One machine with 2015 GB of RAM and four A100 cards generated 0.10 tokens a second, with no measured cause for that result. Unsloth gives about 20 tokens a second on B200s if the model fits, without saying how many GPUs produce that speed.
For a stated busy coding day of 3 million input tokens and 1 million output tokens, Moonshot’s API pricing comes to $16. One estimate puts eight hours of four GPU rental at about $192 using the cheapest listing. At 20 tokens a second, one stream would produce 576,000 output tokens in eight hours, short of the stated day’s output.
A separate K3 benchmark on B200s measured 3,976 tokens a second per GPU while each user got 50. Semi Analysis Inference X measured that while serving many users. It is neither a one-stream speed nor a measurement of K3.1 Pro.
Serving Cost Meets Actual Use
One commentator says 10 million people set up Meta’s Muse in six weeks. The same commentator calls it expensive to serve, says it is not yet a platform, and says Reality Labs lost $19.2 billion in 2025. Alexander Wang, who came to lead Meta’s AI effort, says fewer than 200 people worked across Muse’s model and product.
Another commentator says Elon confirmed that serving Grockbot cost more than it earned, so usage was lowered. In that commentator’s audit, only six of 30 bots were used regularly. They claim removing unneeded routines can cut daily token use by 80%, and suggest reviewing unused bots, group chats, skills, and plugins.
Those are the commentators’ assessments and claims, not measurements from the Mac or K3 tests. Together, the examples keep the engineering question concrete: getting a model running, serving it at a workable cost, and seeing how much of what is served people actually use are separate things.