Engram's RL Training Teaches Agents to Read Their Own Notes 20% Better

Engram’s report describes a short reinforcement learning phase that teaches agents how to use existing text memory. Across three workspaces, training on top of notes improved accuracy by 7 to 20 percentage points and often reduced inference costs.

Agents commonly store knowledge in notes, wikis, and skill files. Effective use requires several judgments: which notes to retrieve, how narrowly to search, when to trust a summary, and when to inspect the underlying files. Engram trained those behaviors after building the notes and optimizing the system prompt.

Three workspaces, one frozen harness

The evaluation covered three distinct environments:

• Harvey C&H: A synthetic law firm containing roughly 100 million tokens of client matters.
• Hex Shorelane Commerce: A synthetic data warehouse evaluated with DataBench.
• Engram Internal: An October snapshot of the company’s Notion workspace.

Engram used Qwen 3.8 27B for Harvey and its internal workspace, and GLM-5.3-Flash for Hex. A separate agent built the notes with Claude Sonnet and Haiku. The baseline system prompt was optimized with GEPA.

Training used Group Relative Policy Optimization, or GRPO, which compares groups of sampled answers and reinforces the higher-reward outputs. LoRA limited updates to compact adapter weights. The reward combined task accuracy with an execution-cost penalty, and changing that penalty produced different accuracy and efficiency tradeoffs.

The training questions were separate from the evaluation sets. During reinforcement learning, Engram froze the notes, workspace, tools, harness, and system prompt, leaving the adapter weights as the only changing component.

Training compounds the value of notes

Trained models beat their corresponding untrained models in all three environments. At several cost settings, they also matched or exceeded larger frontier-model configurations using the same notes.

Two ablations clarify where the gains came from:

• Training without notes: The trained model still beat the base model, but remained behind the version trained with access to notes.
• Alternative note format: Training also improved performance with a more prose-heavy memory recipe, suggesting the effect was not tied to one note structure.

RL costs a fraction of memory setup

Under Engram’s cost model, reinforcement learning cost about 5% to 30% as much as note construction and prompt optimization. Building memory required extensive model inference to explore each corpus and write summaries; LoRA training required a smaller one-time outlay.

Engram calculated break-even time by comparing that training expense with the trained model’s per-query savings. The estimate assumes 20 users making 20 queries each day, for 400 daily queries.

A cost-efficient Harvey configuration matched GPT-6 Sol at medium effort with 78% accuracy. The trained Qwen model cost $0.06 per query, compared with $0.17 for GPT-6 Sol. Actual break-even periods will vary with provider pricing, traffic, task mix, and the selected accuracy target.

Hex exposed a capacity limit. Training improved the smaller model substantially but did not close the full gap to the strongest frontier configuration, which Engram attributes to the base model’s limited capacity for the analysis task.

One training run handles 23M tokens of edits

Memory changes as agents add, reorganize, and rewrite notes, creating a risk that training will overfit to one snapshot. Engram tested this by building Harvey notes from half of the workspace, revealing the remaining material in stages, and updating the notes after each stage.

The final memory had changed by about 23 million tokens, more than three times its original size. Engram trained the model once on the initial snapshot and reused those weights throughout the sequence.

The trained model outperformed the base model at every stage, and its accuracy increased as the notes improved. Retraining from scratch on the final snapshot delivered only a small additional gain. That result supports the interpretation that reinforcement learning taught reusable retrieval and verification strategies rather than memorizing a fixed set of note contents.

The traces reveal a different search policy

Execution traces show several concrete behavioral changes:

• Gap checking: The trained model compared note coverage with the raw workspace after detecting suspicious results, such as a lookup returning fewer records than expected.
• Narrower retrieval: On Harvey, it made 26% more note calls, while each call returned 36% fewer tokens. Focused queries reduced the risk of truncating broad results.
• Source navigation: On Engram Internal, the model followed links from notes to source files more often and relied less on broad searches spanning both stores.

One Harvey trace shows how those habits affected an answer. Asked for the latest White Collar matter with a litigation hold, the base model found that the practice area was absent from the notes, selected the closest listed practice, and returned the wrong matter.

The trained model detected the same gap and ran comm, a Unix utility for comparing lists, against directory IDs and note metadata IDs. It discovered that 133 of 266 matters were missing from the notes, searched the raw files, and found the correct answer.

Where the evidence ends

The benchmark set includes two synthetic environments and one private company workspace, which limits broad claims about production codebases, larger organizations, and other model families.

• Workspace transfer: Training and evaluation used different tasks from the same environment. Transfer across unrelated workspaces remains untested.
• Memory churn: The 23-million-token update experiment covered Harvey only.
• Model capacity: Hex shows that retrieval training cannot guarantee parity with a stronger model on every task.
• Economics: The reported savings depend on sustained query volume and the price gap between the trained model and its frontier comparison.

How to test the approach

A faithful pilot would preserve Engram’s separation between memory construction, training, and evaluation:

1. Build notes and optimize the system prompt for a stable corpus.
2. Create separate training and evaluation tasks that require both note retrieval and source-file inspection.
3. Freeze the notes, tools, harness, and prompt during adapter training.
4. Reward answer accuracy while penalizing inference cost.
5. Compare accuracy, token use, latency, and total cost against the notes-only system and a stronger hosted model.
6. Repeat evaluation after substantial note updates to measure durability.

The work extends research on reinforcement learning for memory and tool use, including Memory-R1 and Search-R1. Engram proposes joint learning of notes and retrieval behavior as the next experiment. Both stages currently pay for workspace exploration, so sharing that process could lower setup costs and adapt the memory structure to the model’s search policy.