MAGE Lets AI Forget People Without Ever Seeing Their Private Data
MAGE lets users request LLM unlearning with just a name instead of uploading sensitive content, mining the model's own memory to build the forget set.
Takeaways
- MAGE performs LLM unlearning from just a name or short description, no user-supplied forget corpus needed.
- It mines the target model's own memorization by iterative probing, building a weighted local memory graph.
- Weighted random walks over the graph generate scoped forget and neighbor QA pairs for fine-tuning.
- Matches or beats external-supervision baselines on TOFU and RWKU across four unlearning objectives.
- Robust to hallucinated QA pairs: memorization alignment matters more than factual accuracy of the forget set.
- Limitations include ~88 elicitation calls per entity and weaker recovery on non-entity attributes.
Machine unlearning for large language models has a dirty secret: to make a model forget something, you usually have to hand over the exact thing you want it to forget. A new framework called MAGE (Memory-grAph Guided Erasure) inverts that assumption. Rather than asking users to upload a corpus of sensitive text, it takes a name or short description and reconstructs what the model remembers about that target from the model itself.
The resulting workflow is easier to audit, less prone to secondary leakage, and, according to the authors, roughly as effective as pipelines fed with hand-curated forget sets.
The trust problem in "right to be forgotten" requests
Regulations like GDPR give individuals the right to request removal of their data from deployed models, and machine unlearning has emerged as a practical way to selectively erase targeted knowledge while preserving overall capabilities. Current pipelines carry an awkward requirement, though.
Most existing approaches rely on a strong and often implicit assumption: the user requesting unlearning must provide a forget set that explicitly contains the information to be removed, which often contains sensitive content. That creates three concrete risks the authors flag:
- Secondary leakage: uploading sensitive material to trigger unlearning can itself expose it if intercepted or mishandled.
- Auditability: service providers cannot easily verify that a submitted forget set actually belongs to the requester or is properly scoped.
- Integrity attacks: seemingly benign forget sets can embed backdoors, which may stealthily degrade utility or make unlearning reversible.

Anchors instead of corpora
MAGE reframes the request. Users provide only a minimal anchor identifying the target entity, such as a name (e.g., "Taylor Swift") or a short description to reduce ambiguity. They do not supply documents, they do not specify an unlearning algorithm, and they do not need any access to the training corpus.