Prime Intellect's ECHO Teaches AI Agents to Predict Their Own Tool Responses
Prime Intellect applies ECHO to teach LLMs to predict their own tool outputs during RL, improving agent generalization and efficiency at near-zero extra cost.
- ECHO applied to agent RL: Prime Intellect tests ECHO, a method that adds a world-modeling loss to standard RL, teaching agents to predict tool outputs.
- Near-zero cost: ECHO reuses the same forward pass as GRPO with no extra rollouts -- just one new hyperparameter,
alpha. - Strong benchmark gains: ECHO doubles GRPO pass@1 on TerminalBench-2.0 (Qwen3-8B: 2.70% to 5.17%; Qwen3-14B: 5.17% to 10.79%).
- Overfitting risk is environment-dependent: Algorithmic tools (code execution) are safe for many epochs; memorization-heavy tools (web search) overfit after ~1 epoch.
- Recovers pretraining capabilities: World modeling during RL measurably lowers Wikipedia perplexity, suggesting it re-activates the model's general simulator.
- Coming to prime-rl: Native ECHO support is planned for the prime-rl open-source library; environments are live on the Prime Intellect platform.
There is a quiet assumption baked into every modern agent training pipeline: the model should learn to act, and the environment's response is just context. Prime Intellect's new research challenges that directly. Their argument is that an LLM that can predict what its tools will return is a fundamentally better agent than one that cannot -- and that you can teach it to do so for free, inside the same training loop you are already running.
The technique they are building on is ECHO (Environment Cross-entropy Hybrid Objective), a method originally proposed by Dimitris Papailiopoulos and colleagues at Microsoft Research AI Frontiers. The core idea is deceptively simple: standard GRPO-style RL (a popular policy-gradient algorithm) only applies a training signal to the tokens the model itself generated -- its actions. ECHO also applies a supervised learning signal to the tool-response tokens that come back from the environment. The model is forced to predict what the terminal, the compiler, or the search engine will return, not just what command to issue next.
One extra loss term, two training objectives
ECHO introduces a hybrid objective that combines the standard policy-gradient loss on action tokens with an auxiliary loss that trains the policy to predict environment observation tokens resulting from its own actions -- and it reuses the same forward pass as GRPO, requiring no additional rollouts.
The implementation trick is elegant: in normal RL training, environment-response tokens are masked out of the loss. ECHO simply assigns them a fixed positive advantage value controlled by a hyperparameter called alpha, turning SFT on tool outputs into a special case of RL with constant advantage. No extra compute, no extra data collection.
CLI agents are the closest thing language models have to an embodied setting -- the model emits commands, the terminal executes them, and the returned stream of stdout, errors, files, and logs records the consequences. Standard agent RL discards this stream: GRPO-style training updates action tokens with sparse outcome-level rewards while ignoring environment responses already in the rollout. ECHO recycles that discarded signal.
What Prime Intellect actually tested
The team ran experiments across two very different environments to stress-test the method:
- forth-lang: A coding environment for Forth, an obscure stack-based programming language that models have rarely seen during pretraining. The model has access to tools for running code, submitting code, and looking up documentation. Crucially, test cases are hidden during rollouts, forcing the model to generalize rather than memorize.
- deepdive: A web-search Q&A environment for training deep research agents, with tools for web search, page scanning, and line-level document reading. Tool outputs here are much harder to predict without memorization.
They evaluated on four environments throughout training: the in-domain test sets for both, plus Wordle (a logic-heavy multi-turn game) and general-agent (a tool-heavy mixed environment). This spread lets them measure whether world modeling in one domain bleeds into others.