Edge Zero reports running a 35 billion parameter model with 2.9 GB of peak memory allocated by MLX.
The checkpoint totals 19 1/2 GB.
Both statements are true. The gap between them explains why a low memory number can look impressive while leaving out much of the machine’s actual work.
2.9 GB Does Not Include the Whole Model
Most of the model’s expert weights stay on the SSD. The operating system can also keep recently used weights in its page cache, and that memory is not included in the 2.9 GB figure.
So the reported number is not the model’s full resource cost. It measures peak memory allocated by MLX, while storage and page-cache memory help keep the model running.
For every token, the model selects four experts across 40 layers. That requires loading 283 megabytes of weights per token.
This is how Edge Zero can keep allocator memory low without fitting the whole checkpoint into that allocation. Most weights remain elsewhere until the system needs them.
The design depends on more than memory capacity. It also depends on how quickly the machine can fetch expert weights from storage and whether spare RAM is available for the page cache.
Predicting Experts Changes More Than Speed
Edge Zero uses 33 small prediction networks to predict which experts the next token will need. The system can then start fetching those weights early instead of waiting until they are requested.
The project reports up to 59% higher decode throughput.
That sounds like a straightforward speed optimization, but the prediction also replaces the model’s original routing. If the prediction is wrong, different experts process the token.
A real failure has already exposed that risk. On the 11th, a user reported that the output had collapsed into exclamation marks. A 16 bit overflow had produced a not-a-number routing value, and the project fixed it the next day.
This is not simply the same model squeezed into a tiny memory budget. Storage and page-cache memory are carrying part of the load, while predicted routing changes which experts handle a token.
That does not make the approach invalid. It does mean the allocator number cannot explain the design on its own.
Low Allocator Memory Can Become Very Slow
Edge Zero’s target is 3.4 GB and 13 tokens per second. The project also reported 113 prompt tokens per second on the first request and 140 on later requests.
Results on constrained machines were very different.
On a memory-constrained 8 GB M1, performance fell to about 0.7 tokens per second. The model card listed 24 tokens per second for that smaller tier.
In the same machine and session, a dense 4B model reached 11.4 tokens per second. A 16 GB MacBook Air using an external USB drive reached about five tokens per second.
These are not interchangeable tests across identical machines, but they expose the practical trade-off. Low allocator memory depends on storage and spare RAM. When RAM is constrained, the design can become page-fault-bound and slow.
The 2.9 GB result therefore does not mean an ordinary low-memory machine gets fast inference from a model of this size. On the constrained 8 GB M1, the reported experience was about 0.7 tokens per second, even though allocator memory stayed low enough to make the run possible.
Quality and Throughput Are Still Hard to Isolate
The quality results also moved in the wrong direction. The project benchmark averaged 79.2, compared with 83.2 for the full-precision base.
On AIME, the scores were:
- Edge Zero: 86.6
- Full-precision base: 92.7
The comparison does not isolate a single cause. It combines 4bit weights, fewer active experts, predicted routing, and a distillation adapter. Any attempt to blame the score gap on one of those changes would go beyond the available evidence.
The throughput claim has not been independently reproduced either. Support is limited to Mac OS and Apple Silicon, and the server handles one request at a time.
That leaves a promising engineering idea with an incomplete measurement story. The allocator figure is real, but so are the 19 1/2 GB checkpoint, the storage traffic, the page-cache memory, the routing changes, and the large slowdown seen on a constrained machine.
The useful next test is an independent reproduction that measures throughput, total machine memory, page-cache use, and long-context performance together.