---
title: "Fudan's Nemori Beats Full Context Memory While Using 88% Fewer Tokens"
subtitle: "A new framework decides what LLM agents should remember based on prediction error rather than hand-crafted heuristics, cutting tokens 88% while beating full-context accuracy."
url: "https://alphasignal.ai/news/fudan-s-nemori-beats-full-context-memory-while-using-88-fewer-tokens"
author: "AlphaSignal Newsroom"
author_url: "https://alphasignal.ai/editorial-team"
company: "AlphaSignal"
company_url: "https://alphasignal.ai/company/alphasignal"
published: "2026-04-16T00:00:00.000Z"
updated: "2026-08-23T21:00:04.257Z"
type: "Paper"
topics:
  - "Agents"
  - "Post Training"
subtopics:
  - "Memory"
  - "Distillation"
upvotes: 3273
source_url: "https://arxiv.org/abs/2508.03341"
comments: 0
truncated: true
---

# Fudan's Nemori Beats Full Context Memory While Using 88% Fewer Tokens

> A new framework decides what LLM agents should remember based on prediction error rather than hand-crafted heuristics, cutting tokens 88% while beating full-context accuracy.

Most memory systems for LLM agents decide what to store using rules the designer wrote by hand: importance scores, emotional tags, or factual templates. A new framework called [Nemori](https://arxiv.org/abs/2508.03341) throws that playbook out and lets the data itself decide, borrowing an idea from neuroscience: if the agent could already predict what happened, it is not worth remembering.

The approach comes from researchers at Fudan, Shanda Group, Beihang, and Shanghai University of Finance and Economics. On the LoCoMo long-conversation benchmark, Nemori edges out feeding the entire dialogue history into the context window while using roughly one-eighth the tokens. On the much longer LongMemEval benchmark, the gap widens dramatically.

## The heuristic trap

Memory systems for LLM agents identify useful experiences through two stages: distillation, which determines the entry form of experiences, and management, which handles ongoing maintenance. Existing systems mostly intervene at the management stage, tracking access frequency, temporal decay, or graph links, or bake designer intuition into distillation.

Heuristics risk introducing subjective bias, which is fatal during distillation because it causes irreversible information distortion. To avoid that distortion, systems tend to over-store, which then amplifies retrieval noise. Once you decide to keep everything just in case, retrieval quality collapses.

## Prediction error as a signal

Nemori draws on Predictive Coding Theory from visual neuroscience, which frames the brain as a prediction machine that only propagates surprises upward. Applied to agent memory, prediction error signals information worth retaining; whatever is predictable is redundant.

For each new chunk of conversation, Nemori asks its existing knowledge base to guess what happened. The gap between the guess and what actually happened becomes the semantic memory. If the model could already predict it, there is no reason to write it down.

_Preview only. Full text requires an AlphaSignal Pro account._

## Sources

- [What Deserves Memory: Adaptive Memory Distillation for LLM Agents](https://arxiv.org/abs/2508.03341)
- [nemori-ai/nemori GitHub Repository](https://github.com/nemori-ai/nemori)
- [Nemori HTML paper on arXiv](https://arxiv.org/html/2508.03341)