A-EVO Lab's Adaptive Auto-Harness Stops AI Agents From Getting Worse Over Time
Adaptive Auto-Harness solves the 'peak early, then decline' failure mode of self-improving LLM agents on real, never-ending task streams
Takeaways
- Core problem: Self-improving LLM agents peak early then degrade -- a single harness overfits to early tasks and fails as distributions shift.
- Solution: Adaptive Auto-Harness uses a harness tree with solve-time routing, a stateful multi-agent evolver, and human-steering hooks to sustain improvement.
- Key results: 80.9% accuracy on PolyBench vs. 50.8% for the next best baseline; also leads on CTF-Dojo (50.2%) and FutureX (49.5%) forecasting.
- Three failure modes addressed: Unbounded history, task heterogeneity, and non-stationarity -- none of which offline benchmarks expose.
- Architecture insight: Evolution (background) and routing (inference-time) are decoupled, so live tasks are never blocked by ongoing harness updates.
- Open source: Code available at github.com/A-EVO-Lab/AdaptiveHarness.
Self-improving AI agents have a dirty secret: they tend to get better for a while, then get worse. The culprit is a pattern researchers call the dense harness problem -- as an agent accumulates experience and updates its scaffolding (prompts, tools, memory, routing logic) from that experience, it eventually overfits to early tasks and starts degrading on new ones. A new paper from A-EVO Lab introduces Adaptive Auto-Harness, a framework specifically designed to prevent this collapse and keep agents improving indefinitely on open-ended, real-world task streams.
What is a "harness" and why does it matter?
Before diving in, a quick definition: a harness is everything wrapped around an LLM that isn't the model weights themselves. It includes the system prompt, tool definitions, memory retrieval logic, routing rules, and orchestration code. Changing only the harness around a fixed LLM can produce a 6x performance gap on the same benchmark -- which means harness design is often more impactful than the model you pick.
Auto-harness systems such as A-Evolve, GEPA, and Meta-Harness improve LLM agents by optimizing prompts, skills, tools, memories, and supporting infrastructure from execution feedback, but they are typically evaluated on fixed offline benchmarks. That's a critical gap. In production, tasks never stop arriving, they come from different domains, and the distribution of what you're asked to do shifts over time.

Three problems that break existing systems
The paper identifies three concrete failure modes that arise in real deployments -- none of which offline benchmarks can expose:
- Unbounded history (D1): Task histories grow without a fixed endpoint, eventually exceeding any context window. A harness optimized on the first 100 tasks can't be naively re-applied to task 10,000.
- Heterogeneity (D2): A single monolithic harness can't be good at everything. A harness tuned for prediction markets will be mediocre at security competitions.