Dream-RSI replays coding-agent histories to tune exploration

Researchers at the University of Maryland, the University of Virginia, Google, and Google DeepMind have released the Dream-RSI preprint, short for Recursive Self-Improvement through Evolving Worlds. Lead author Tong Zheng hosts the PDF through a personal GitHub repository. A project page is also live, while the codebase, discovered programs, and reproduction scripts remain unavailable.

Why the outer loop gets expensive

Self-improving coding systems often use an outer policy to control how an agent explores candidate programs. That policy chooses which branches to pursue, how many workers to run, and when to stop. Comparing policies usually requires fresh agent calls and code executions across thousands of iterations, so weak exploration strategies can consume substantial compute before producing useful results.

Dream-RSI reduces that evaluation cost by reusing completed discovery runs. The method targets the exploration policy around the coding agent rather than retraining the underlying model.

Turn traces into replay worlds

Each completed run produces a structured tree containing exploration decisions, generated programs, and execution results. Dream-RSI compiles those trees into replay simulators. Candidate policies can traverse recorded branches in different orders, select different subsets, alter parallel groupings, and test stopping rules while retrieving cached outcomes.