Engram's Legal Agent Beats Claude Opus 4.8 at One-Tenth the Cost

The AI memory startup Engram just dropped its first public research report, and it takes direct aim at one of the most expensive habits of modern agents: rereading the same documents every single time a user asks a question. Working with legal AI company Harvey, the team built a synthetic law firm and trained an agent to study that workspace before deployment, baking knowledge into both text notes and model weights.

The headline result is striking. Their agent hits $0.13 per query with a 30% all-pass rate, compared to $1.32 per query and 25% all-pass for Claude Opus 4.8. That is roughly an order of magnitude cheaper while scoring higher, and it comes from a much smaller open base model (Qwen3.8-27B) rather than a frontier system.

The problem with stateless agents

Legal work is a nasty stress test for retrieval-augmented setups. The synthetic firm, Calderwood and Harkness, contains 100M tokens across 266 matter folders and 9,286 files. A single realistic question, like finding the most recent M&A deal with a working-capital adjustment, can require an agent to skim hundreds of documents. For Opus 4.8 at high reasoning, the average trajectory costs $1.32 and only succeeds 25% of the time, with long runs regularly exhausting the context window.

Lawyers, of course, do not reread every file for every question. They remember. Engram's argument is that agents should too, and that memory should live in more than one place.

Three kinds of memory, one training loop

The recipe combines three complementary stores that get trained end-to-end:

• Parametric memory baked into the weights, good for diffuse associations like the shape of a multi-party contract.
• Text notes the agent writes to itself, good for structured, editable facts like closing dates and case statuses.
• Search tools over the raw filesystem for anything the first two miss.

During training the agent explores the workspace through thousands of rollouts, writes text notes to organize what it finds, and self-generates training data to internalize the workspace. Lightweight knowledge modules are trained via a mix of distillation on model outputs and reinforcement learning on model-generated problems. The agent compresses 100M tokens of raw content into roughly 1M tokens of notes, and it discovers useful organizing features like practice areas and case statuses on its own, without being pointed at the evaluation questions.

Under the hood, Engram trains on top of open-source base models using parameter-efficient finetuning, with plans to explore combinations of LoRA, trainable KV caches, and added experts in MoEs.

What changes in the agent's behavior

The interesting part is not just the cost curve, it is how the trained model acts. Compared to the base Qwen3.8-27B, the studied model behaves like someone who has actually worked at the firm:

• It begins with a targeted search in 99.2% of trajectories, versus 3.2% for the base model, which usually opens its notes first.
• In 24.5% of first turns, its thinking trace already names at least one correct matter, versus 0% for the base.
• It uses 9.16 new filter terms per rollout versus 2.43, and returns 5.59 unique matters per search versus 12.17, indicating narrower, more informed queries.
• It reaches the first correct matter in 3.73 turns instead of 5.94.

To confirm the knowledge is actually in the weights and not just in the notes, the team ran closed-book probes where the model answers questions with no tools and no memory access. Closed-book accuracy rose from 4.7% to 72.6% between training steps zero and one thousand. In one striking demo, the model was asked to predict the exact output of a complex jq shell command over its notes, and produced a 100% character-for-character correct answer, even applying tac to reverse the line order, despite never being directly trained on tool-call outputs.

Where this fits in the bigger picture

Engram is not a random lab. The company came out of stealth with $98M from General Catalyst, Kleiner Perkins, Sequoia, and angels including Andrej Karpathy, and counts Microsoft, Notion, and Harvey among its customers. The founding thesis is that the constraint holding AI back is memory, not reasoning power, and this legal-firm report is the first concrete evidence they can back the pitch.

The framing worth internalizing is that studying shifts compute from inference time to training time, which makes sense when many queries hit the same workspace repeatedly. For any team running agents against a stable corpus, whether that is a legal filesystem, a monorepo, or an internal wiki, the traditional RAG pattern of stuffing retrieved chunks into context on every call starts looking wasteful. The tradeoff becomes: pay once to teach the model your organization, then amortize that across thousands of cheaper, sharper queries.

Notable caveats remain. The benchmark is synthetic, the base model is a specific 27B dense checkpoint, and Engram has not open-sourced the training recipe. Real workspaces also change constantly, and the team flags continual learning, adapting the study regimen as query distributions and documents shift, as an open research problem. Still, the direction is one of the more concrete signals yet that the next efficiency frontier for agents may not be a better retriever or a longer context window, but a model that actually remembers where it works.