GLM5 under Open Code rose from 64.2 to 75.7 on Workbuddy at K=1, according to Harness Squared’s authors. The model weights stayed frozen. The reported improvement came from changing the agent’s setup: the instructions and other material that shape what it does with the model.
That distinction is the story here. A higher score for an agent does not mean the model inside it has changed. Harness Squared reports a better result from changing how that model was used. The size of the gain is interesting, but so are the extra runs and the access conditions behind it.
The Setup Was the Part Harness Squared Edited
Harness Squared can edit an agent’s instructions, memory, skills, tool guidance, scripts, sub-agents and plugins. Those are different parts of the setup, but they all sit around the model rather than changing its weights. The reported GLM5 result asks us to pay attention to that surrounding work.
For each task, the system tries two proposed edits on the same task and model. One addresses recurring problems, such as tool use. The other addresses knowledge and procedures for that particular task. Harness Squared then compares what the agents did and produced, and can combine the edits or keep the original setup.
The same-task comparison matters. It lets the authors judge proposed edits against the setup they started with, without treating a different model as the explanation for the change. But the result still belongs to the configured agent. The score alone cannot tell a reader which part of the setup contributed most.
The Authors Report Gains on Workbuddy and Code Tasks
The authors report these results:
- GLM5 under Open Code, Workbuddy at K=1: 64.2 → 75.7
- Gemini 3.5 Flash under Open Code, Workbuddy at K=1: 72.4 → 77.5
- GLM5, code tasks: 58.4 → 77.1
The Workbuddy result for GLM5 is the clearest example of the article’s central point: the reported score rose while its weights stayed frozen. The Gemini 3.5 Flash result gives the authors another Workbuddy gain, and they also report a gain for GLM5 on code tasks. Each figure belongs to its stated model and task; none is a blanket score for agents built this way.
Taken together, the reported results make the agent setup worth examining. They also leave a practical question about what it took to get those results. Harness Squared does more than run an agent once with revised instructions. It proposes changes, runs comparisons, and uses additional calls to edit and grade them.
More Runs, With Broad Access
In the paper, one K=1 step counts as three agent runs. Count the composer’s run and it becomes four. Editing and grading calls come on top. Those counts belong beside the scores because they describe the work involved in reaching a reported gain.
The cited results have no independent reproduction. The README says the executor had unrestricted network access and Codeex ran with its sandbox disabled. In one documented run, Codeex downloaded Jobbench’s public rating rubric without being told to.
That documented download does not erase the reported scores. It does make the access conditions part of how to read them. A reader judging the gain needs to know both what changed in the agent and what the agent could reach during the run.
The authors have reported an interesting way to improve agent results while leaving model weights alone. The remaining question is specific: can independent tests reproduce those gains with tighter access controls?