- With
GPT-4o-mini, building theMem0store cost $0.56, more than answering all 152 questions in full context at $0.43, and the same history breaks even after 372 to 596 questions. - Claude Opus 5.5 reached 95% with
Mem0and with full context, whilePion that model scored 99% for $6.95 against $17.84 for full context. - A
Zepgraph written byGPT-4o-miniscored 43% for that model and 45% for Claude Opus 5.5, then 87% when Opus wrote the graph and read it. Piread about 10,000 input tokens per question against 28,900 for full context, and on the 88 questions with no photo evidence it scored 100% against 99%.- On Claude Opus 5.5,
Codexcost $25.41 at 98% and Claude Code cost $28.41 at 97%, againstPiat 99% for $6.95.
AI agents need memory to handle long-running tasks. But adding a memory layer isn't free. Extracting facts, building knowledge graphs, and retrieving past information all come with costs. And sometimes, these systems can make agents less effective.
A new study puts seven memory approaches to the test, raising important questions about how developers should design their agent stacks.
Today we break down the hidden memory tax and when it's worth paying.
The hidden memory tax in AI agents
AI agents might carry out sessions or conversations that last for a long time. For example, a customer-support agent might interact with a client for six months.
When asked about an earlier purchase or promise, should it search the original conversations or rely on a structured memory system that extracts important details? The answer affects the agent's accuracy, inference costs and maintenance.
MemoryTax, a new study by Shanghai Jiao Tong University, tests seven ways agents can access past conversations. The findings suggest that weaker models can benefit from structured memory but stronger models often do better when given the raw information.
For developers, the question is when building a separate memory layer earns back its cost.
What MemoryTax tested
The study used one conversation from LoCoMo, a benchmark for long-term conversations with hundreds of turns. The experiment tests the agent's ability for factual recall, connections across messages, and the sequence of events.
They tested seven different memory approaches paired with three models: GPT-4o-mini, GPT-6 Luna, and Claude Opus 5.5.
The seven approaches fall into several groups:
- Full context: The entire conversation is inserted in the model's prompt. This avoids building a separate memory store but repeatedly processes the same history.
- Retrieved memory: The system uses an algorithm like BM25 to search the raw history and retrieve the parts that are relevant to the question.
- Precomputed memory: These are memory consolidation techniques like Mem0, which uses a model to extract and update useful facts, then retrieves them later. Another is
Don't miss what's next in AI
Join 300,000+ engineers and researchers who get the signal, not the noise. Create a free account to read the rest of this story.
- Full access to in-depth AI research breakdowns
- Be the first to know what's trending before it hits mainstream
- Daily curated papers, repos, and industry moves
Comments
