Context Rot Is a Harness Problem, and RLMs Show Why
New MIT CSAIL work shows generalization can live in the harness: train short, generalize 8-32x longer. Here's the test to run on your agent first.
- Trained inside a Recursive Language Model harness,
Qwen3-30B-A3B-Instruct-2507generalized from short training tasks to held-out tasks 8-32x longer, while the same weights trained as a plain Transformer stayed flat on the long evaluations. - The gains are not free: RLM training runs 1.5-3x longer than the plain Transformer, and on MRCRv2 the model first collapsed to a single giant sub-call until a
nudge to decomposepushed it toward the general strategy. - Context rot is a distribution problem rather than a capacity one: Chroma tested 18 frontier models and found quality drops as input grows, often before the window is full, a decay that append-only loops like
ReActandClaude Codemake worse. - Even before any training, an off-the-shelf RLM beat common long-context scaffolds against GPT-5 by a median of 26% over compaction and 13% over Claude Code (
arXiv:2512.24601). - The harness also transferred across domains (same-author essays to same-solution math), scored by
nDCG@10, and you can start withdspy.RLMatmax_iters=20andmax_llm_calls=50, or therlmspackage.
Your coding agent runs clean for twenty minutes. Then it starts forgetting what it read three tool calls ago, and the obvious move is to reach for a bigger model.
That reflex usually aims at the wrong layer. New research from MIT CSAIL's Alex Zhang and Omar Khattab argues that the thing which generalizes to harder, longer tasks often isn't the model at all, but the harness around it.
Their headline result: a model trained only on short tasks generalizes to tasks 8-32x longer once it sits inside the right harness. Before you upgrade anything, it's worth knowing whether your bottleneck is the model or the scaffolding.
What context rot actually is
Context rot is quality falling as input grows, often well before the context window is full. Chroma tested 18 frontier models and found every one degrades as inputs get longer, sometimes far short of the advertised limit.
Standard agent loops make this worse by design. ReAct, Claude Code, and Codex append every observation, tool output, and reasoning step into one growing prefix, and that prefix drifts out of the distribution the model trained on.
More context is not the same as usable context. A failure that looks like a weak model reads, on inspection, as a distribution problem: the inputs have wandered off anything the model saw in training.
Zhang and Khattab give the good case a name, locally in-distribution (LID). A well-built harness keeps every individual model call in-distribution even when the overall task is not.
How a harness can carry generalization instead of the model
A Recursive Language Model (RLM) carries the generalization by never handing the raw problem to the model. Long context lives as a variable in a code environment, and the model writes code to examine, slice, and query it.
Two moves do the work. Context offloading keeps the bulk input out of the root prompt, and programmatic sub-calls store their outputs in variables the root model never has to re-read.
”The two moves that keep the root model in-distribution: offload bulk context to a variable, and keep sub-call outputs in the REPL instead of the transcript. Source: Zhang & Khattab, 2026.”
The payoff is subtle: two tasks that look nothing alike on the surface end up producing near-identical code at the root, so a model trained on one has effectively trained on the other. Zhang and Khattab call such tasks isomorphic.
Picture two jobs: tally how many of 200,000 support tickets mention refunds, and count how many lines in a giant server log are errors. Different domains, yet the root model writes the same program for both, chunking the data, fanning out a sub-call to label each piece, and adding up the results, with only the sub-call's wording changing between them.