MemPalace Hits 96.6% on the Hardest AI Memory Benchmark Without LLM Calls

MemPalace hits 96.6% on LongMemEval with zero API calls, using verbatim storage and local vector search — no cloud, no subscription, no LLM at write time

·
·
·
MemPalace Hits 96.6% on the Hardest AI Memory Benchmark Without LLM CallsPRO
Read2 min
TypeRepo
SubtopicMemory · Mcp
  • Record benchmark score: MemPalace hits 96.6% R@5 on LongMemEval with zero API calls, using only ChromaDB and verbatim text storage.
  • Free and local: MIT-licensed, runs entirely offline with just two dependencies (ChromaDB + PyYAML) — no subscription, no cloud, no API key required.
  • Core insight: Storing conversations verbatim and solving retrieval at read time outperforms LLM-based extraction at write time — challenging the entire field's default approach.
  • Benchmark controversy: Independent auditors found metric mismatches (retrieval recall ≠ QA accuracy) and a LoCoMo score inflated by setting top_k=50 on a 19-item dataset; maintainers corrected all claims publicly.
  • Competitive gap narrowing: Mem0's April 2026 token-efficient algorithm raised their LongMemEval score from ~49% to 93.4%, but MemPalace remains the top free, offline option.
  • Get started: pip install mempalace — full docs at mempalaceofficial.com and GitHub repo.

MemPalace is an open-source AI memory system built around a deceptively simple idea: stop trying to be clever about what to remember, and just store everything. No AI-driven extraction. No summarization. No lossy compression. Just verbatim conversation text, indexed with vector search, organized into a hierarchical structure inspired by the ancient mnemonic technique known as the memory palace. The result is a system that scored 96.6% on LongMemEval , the hardest long-term memory benchmark in the field , without making a single API call.

The problem every AI app hits eventually

Every transformer-based LLM starts each conversation from zero. Semantic vector search retrieves the right memory in milliseconds, and this is what drives the benchmark-topping 96.6% score. But getting to that retrieval requires solving a harder problem first: what do you store, and how?

The dominant approach across the industry has been extraction-based: use an LLM to read your conversations, pull out the important facts, and store those. Every competitive memory system uses an LLM to manage memory, all starting from the assumption that you need AI to decide what to remember. The problem is that this decision is made at write time, before you know what questions will be asked at read time. Any extraction is a bet on future relevance , and those bets lose information.

MemPalace bets the other way. It stores your conversation history as verbatim text and retrieves it with semantic search. It does not summarize, extract, or paraphrase. The benchmark results suggest this is not just philosophically cleaner , it's empirically better.

Pro article

This story is for Pro members

You've reached the end of the free preview. Upgrade to AlphaSignal Pro to read the full article - and everything else behind the paywall.

Trending
  • No trending articles

Comments

avatar

Next Reads