OpenAI released solutions to open math problems and put 722 manuscripts on GitHub. Anyone can scrutinize the work there. Posting it, though, does not vet the manuscripts or peer review them.
That gap runs through the rest of today’s AI story. A model can report a score, an agent can finish a task, and an editing tool can save a file. Each result tells us something. The harder question is whether it tells us what we need to know.
722 Manuscripts Still Need Review
@CalebWritesCode reported more than 40,000 research submissions in September. A speaker also described journals limiting how often researchers can submit work as AI-assisted submissions rise. More work can be shared, but deciding which results are correct and worth pursuing remains the hard part.
@NateBJones described Tao’s comparison with a medical treatment that works even when its mechanism is unknown. The speaker responded that medicine sometimes uses treatments without fully understanding how they work, and acknowledged that such treatments work outside both medicine and math. It is a comparison about judging a result. It does not check a proof.
The Score Depends on Which Model You Can Use
Meta shipped two models under the Muse Wonder name. Artificial Analysis scored Full at 56, the API model at 52, and Claude Opus 5.5 at 58. Both Muse Wonder scores used max effort.
Full is available through Muse Code. Developers using Meta’s API get the other model, so Full’s score does not describe the model they can call. That distinction matters before anyone compares the scores.
The API model cost $4.52 per index task, compared with $3.26 for GPT-6 Astra. It generated output at 115 tokens a second, but took about 20 seconds to produce its first token, against an average of 5.52 seconds. Output was quick once it began; the wait before it began is part of the experience too.
Meta also says Muse Wonder has a 1 million token context window, active memory, and active focus for longer jobs. Its published material does not explain whether active memory protects a constraint during one long run or give active focus scores. An earlier SWE Marathon study found frontier agents solved under 30% of 20 multi-hour software tasks. Those agents predate Muse Wonder, so that result is a reason to test long jobs, not a Muse Wonder score.
A proposed llama.cpp change offers another example of a result with a defined scope. MTP drafts guess upcoming tokens, and the target model checks them together. The change lets a draft sample possible tokens instead of always picking its most likely one; the PR says this preserves normal sampling behavior.
On an RTX 5090 and a DGX Spark, the PR’s tables report average MTP throughput gains of 3.78 to 9.18% at temperatures 0.5 to 1.0. The best individual result was +12.91%. At temperature zero, results ranged from -3.22 to +5.20%. That is a useful speed result within the PR’s tests; whether the gains carry over to prose is still open.
More Ways to Run Work Still Leave Results to Check
Claude cloud sessions clone a GitHub repo into a virtual machine and work on their own branch. They can continue while the user’s laptop sleeps, and the phone app can surface a question from a running session. AI Tweets also reported a Claude Code Desktop sessions hub with live status and Codex CLI support for parallel tasks in isolated codebase copies and branches.
These are ways to run, organize, and observe work. They do not, on their own, tell us whether it was done correctly. In a separate tutorial, @aicoachjohn covered memory, current state, retrieved information, tools, and orchestration around an agent. The practical evaluation advice was specific: after changing a prompt, rerun the same 50 questions.
A File Can Save and Still Look Wrong
PhotoCraft reported passing 258 of 258 Photoshop files in an open, save, and reopen test. In a closer comparison with Photoshop’s rendered image, it passed at least 130 of 256. Both results matter, but they answer different questions. A PSD can contain a finished composite image to compare against, while that image does not show how Photoshop produced it.
A creator’s Higsfield Katana tests through Claude made the need to inspect output concrete. One video remake kept the requested mask and changed the name line, but it also changed the visor color and used substitute fonts. That run cost 43 credits.
A head replacement needed two retries and cost 352 credits. It redrew frames in ways that could change clothes and room details. A car edit reported 242 credits for its first run and no new credits for a revision.
There was also a billing question. The three run summaries totaled 637 credits, while the account balance fell by 643.5. The summaries did not explain the difference.
The edits worked, but their output details and credit balance still needed checking. For the math manuscripts, the checking task is harder to settle: which results are correct, and which are worth pursuing?