RRSI Tries to Stop Agent Harnesses From Learning the Test

When an LLM repeatedly rewrites its prompts, tools, memory, and control flow to raise a benchmark score, the surrounding agent system can encode quirks of that evaluation. The resulting gains may disappear on unseen tasks. Researchers from Google Cloud AI Research, Stanford, UNC-Chapel Hill, and Washington University in St. Louis describe this as statistical overfitting and propose a regularized search method called RRSI.

RRSI, short for Regularized Recursive Self-Improvement, applies regularization to the agent harness while leaving the backbone model frozen, meaning its weights do not change. The harness includes system and task prompts; planning, action, reflection, and stopping logic; tool interfaces and descriptions; memory and skill files; and the context management that determines what the model sees at each step. The researchers describe the method in a new paper.

Automated harness search asks an LLM proposer to generate candidate edits, evaluates each candidate on an optimization suite called the evolve set, and carries the winner into the next round. Accepted changes become the starting point for later proposals, making the process recursive. Separate out-of-distribution benchmarks measure whether those changes transfer to tasks excluded from the search.

When Harness Search Learns the Benchmark

Finite evolve sets expose automated harness search to three failure modes:

• Benchmark-specific fitting: Prompts, memories, or control logic encode task names, answer patterns, or other evaluation-specific details.
• Noise chasing: A candidate wins because repeated evaluations vary, rather than because the harness improved reliably.
• Complexity accumulation: Edits add tokens, tools, or instructions without producing durable gains.

On agentic workspace tasks, where agents use tools to complete multistep work across files and applications, several existing search methods failed to improve unseen-task performance. Meta-Harness led the baselines on the evolve split but added only 0.9 points to the out-of-distribution average.