Context Rot Is a Harness Problem, and RLMs Show Why

New MIT CSAIL work shows generalization can live in the harness: train short, generalize 8-32x longer. Here's the test to run on your agent first.

Your coding agent runs clean for twenty minutes. Then it starts forgetting what it read three tool calls ago, and the obvious move is to reach for a bigger model.

That reflex usually aims at the wrong layer. New research from MIT CSAIL's Alex Zhang and Omar Khattab argues that the thing which generalizes to harder, longer tasks often isn't the model at all, but the harness around it.

Their headline result: a model trained only on short tasks generalizes to tasks 8-32x longer once it sits inside the right harness. Before you upgrade anything, it's worth knowing whether your bottleneck is the model or the scaffolding.

What context rot actually is

Context rot is quality falling as input grows, often well before the context window is full. Chroma tested 18 frontier models and found every one degrades as inputs get longer, sometimes far short of the advertised limit.

Standard agent loops make this worse by design. ReAct, Claude Code, and Codex append every observation, tool output, and reasoning step into one growing prefix, and that prefix drifts out of the distribution the model trained on.

More context is not the same as usable context. A failure that looks like a weak model reads, on inspection, as a distribution problem: the inputs have wandered off anything the model saw in training.

Zhang and Khattab give the good case a name, locally in-distribution (LID). A well-built harness keeps every individual model call in-distribution even when the overall task is not.

How a harness can carry generalization instead of the model

A Recursive Language Model (RLM) carries the generalization by never handing the raw problem to the model. Long context lives as a variable in a code environment, and the model writes code to examine, slice, and query it.

Two moves do the work. Context offloading keeps the bulk input out of the root prompt, and programmatic sub-calls store their outputs in variables the root model never has to re-read.

”The two moves that keep the root model in-distribution: offload bulk context to a variable, and keep sub-call outputs in the REPL instead of the transcript. Source: Zhang & Khattab, 2026.”

The payoff is subtle: two tasks that look nothing alike on the surface end up producing near-identical code at the root, so a model trained on one has effectively trained on the other. Zhang and Khattab call such tasks isomorphic.

Picture two jobs: tally how many of 200,000 support tickets mention refunds, and count how many lines in a giant server log are errors. Different domains, yet the root model writes the same program for both, chunking the data, fanning out a sub-call to label each piece, and adding up the results, with only the sub-call's wording changing between them.