pull down to refresh
As they use the harness, they feed the traces and output to what they call a proposer which proposes patches to the modifiable surface of the harness (the stuff that @optimism lists), they then evaluate the results produced by each patch relative to the pre-patch harness using a "hidden selection set" of problems, accept or reject each patch based on whether it improves the harness performance or not, then at the end of that loop make sure it's not overfitting to the hidden selection set by testing it against a final "holdout set."
They have a cool picture of the process.
You read that as ongoing metrics against a the last pre-patch run?
nvm the caveat that I missed is the fixed task.
Somewhat - but I understand it to be not as freeform as what Claw does. From the paper:
StarHarness studies a narrower deployment problem: adapting a frozen model’s harness to a stateful enterprise environment. Its search space includes prompts, tools, skills, MCP providers, subagents, and execution policy.
It's a bit hard to say how they do it exactly with the source still under soon™, but I'd imagine they A/B test prompts/skills/mcp and tool source code, and then optimize and generalize what works into skills and subagent templates.
What does evolving harness mean? Like the AI rewrites its own harness?