Google Lets AI Agents Rewrite Their Own Code at 10x Lower Cost
A Google framework treats the agent runtime as mutable code, showing modular self-edits with validation gates beat full rewrites on SWE-bench Lite at a fraction of the token cost.
Takeaways
- Google researchers formalize self-improving agent harnesses as four editable runtime components managed by a four-stage diagnose-propose-validate-deploy loop. Paper
- On SWE-bench Lite, modular gated evolution hits 40% held-out accuracy versus 5% for the seed harness.
- Prompt-only reflection regressed accuracy from 5% to 2.5%, showing text rules cannot replace runtime code hooks.
- Unconstrained full-harness rewrites reached 42.5% but burned 451.6k tokens per task, nearly 10x the gated approach.
- Evolved harnesses transfer zero-shot to a stronger Gemini model, with modular edits generalizing better than longer turn budgets.
- The paper catalogs five failure modes including prompt bloat, benchmark cheating, and evaluator tampering that self-modifying agents must be defended against.
Google researchers argue in a new paper that the software surrounding a language model is a major lever for agent reliability and cost. Their framework treats this software, called the agent harness, as mutable code that an optimizer can revise. In a 60-task experiment, scoped module edits delivered nearly the accuracy of a full rewrite while using about one-tenth as many tokens.
An agent harness turns next-token prediction into sustained execution. It assembles and compacts state, schedules turns, runs tool and shell calls in sandboxes, and checks results before allowing a task to finish. Models vary in their tool-call formats, instruction-following behavior, and stopping decisions, so a harness tuned for one model can perform poorly with another. Manual adjustments can also accumulate regressions as the runtime grows more complex.
Four knobs, four stages
The framework divides a harness into four independently editable components:
- Context and state: Builds the model’s working history and compacts older observations.
- Agent control: Manages the turn loop, retries, stopping rules, and subagents.
- Action interface: Exposes tools, file operations, and sandboxed shell commands.
- Verification and recovery: Checks outputs, runs tests, and resumes work after recoverable failures.
An outer optimization loop updates those components through four stages:
- Diagnose recurring failures from execution logs and trajectories.
- Propose a scoped code or hook patch.
- Validate the candidate on tasks excluded from optimization.
- Deploy the update or route tasks to a model-specific harness.
Failure attribution determines whether the optimizer should edit the harness. If the model identifies the correct fix but an edit tool rejects the patch format, or the control loop exits before tests run, the runtime receives the blame. If tool calls execute correctly and repeated attempts still fail on domain logic, the optimizer abstains. That distinction keeps search compute focused on defects the harness can address.