In the Torch TPU paper, 98% of nearly 5,000 popular Hugging Face models ran. That makes trying a PyTorch model on a TPU sound simpler. The same paper reports a 30 to 40% end-to-end cost for one safer way of handling jobs whose ranks run different code.
Those results measure different things. Broad compatibility tells you a model can run. It does not tell you what a particular job will cost.
Most Models Ran. The Workload Question Remains.
Torch TPU puts the TPU connection into PyTorch’s normal way of sending operations to a device. It covers about 450 core operations, so a tensor on a TPU can remain an ordinary PyTorch tensor. The project’s paper also reports that 97% of architecture families matched GPU numerics.
Google says a mode that groups operations into a program is 50 to over 100% faster than its one-operation-at-a-time mode. That is a useful speed result, but the comparison is between two Torch TPU modes. It does not say how the same PyTorch training job would run on an Nvidia GPU.
The paper says training and generation on the same chips for reinforcement learning is not supported out of the box. There is also no same-job PyTorch training comparison against Nvidia GPUs here, and no chip-price comparison. Torch TPU makes a wide range of models easier to try; choosing it for a workload still requires answers the compatibility figure cannot provide.
The Local AI Winner Changes With the Test
Nanbeige’s 33B model and Qwen’s 27B model trade places in Nanbeige’s own coding table:
- Deep Suite: 65.5 to 42.5 for Nanbeige.
- Swebench Pro: 61.7 to 60.3 for Qwen.
The test harness changes between those rows, and the table does not explain the reversal. If you kept only one score, you would miss both the change in winner and the change in test.
A 24GB GPU raises a different question: what fits once a conversation starts? The models’ 4-bit files are 20.8 GB and 17.4 GB before that conversation consumes more memory. One commentator estimates the growing portion at about 208 kilobytes per token for Nanbeige, versus about 64 kilobytes for Qwen. In that estimate, only 16 of Qwen’s 64 layers keep the growing record.
Speed has the same measurement problem. A GPU Unlock readme reports 14 to 36% prefill gains on RTX 3000 cards. During decode, as the answer came out token by token, the reported difference was half a percent; one inference run was 10 to 12% faster overall. The prefill gain mattered in that test, but it was not the speed gain for the whole run.
A Finished Output Can Leave the Hard Part Behind
In a sponsored Higsfield video test, four still images planned a chain reaction on a desk. The first two takes moved the camera between views. For the third, the tester rebuilt the frames from the same wide shot and asked for a static camera. It stayed still for the full clip.
The frames also carried an awkward action into the result. A duck landed beside a notebook before hopping onto it; an Edit Video request then replaced the duck with a robot. A frame-by-frame comparison found that about 1 and a half percent of an average frame changed, with almost nine in ten changed pixels around the robot. Much of the scene stayed put, but the robot kept the duck’s path and hop.
One commentator judges agent assignments by the work left for a person when the agent finishes. A report still needs reading and a decision, and making the report recurring makes that demand recur. In another example, a painting contractor caught Claude using a floor-area multiplier when a bid needed actual wall measurements.
The commentator advises exploring more ideas than you commit to, ending recurring assignments, and having an agent that coordinates work account for the person’s attention. The video test and the agent examples put a practical limit on “finished”: a completed output can still carry a choice or correction that belongs to a person.
The Buildout Has a Debt Question
The IMF counts $650 billion of debt in deals among AI companies. In deals it reviewed since 2023, debt and equity came with promises of $1.4 trillion in future revenue. The IMF warns that if one company’s AI profits disappoint, its lender and supplier can both feel it.
One commentator points to Meta’s Hyperion financing to explain the structure. Funds run by Blue Owl own most of the Louisiana data-center venture; Meta rents the buildings and guarantees part of their value. The commentator sees a real financing risk, while saying the timing of any crash cannot be checked.
Nvidia’s latest filed quarter showed nearly $60 billion of profit, with results running only to July. The buildout involves real equipment and buildings. The IMF’s question is who carries the debt if expected AI revenue falls short.