Fudan's Mirage Catches AI 'Forgetting' Tools Hiding Data in Plain Sight
A new auditing framework called Mirage exposes that vision models certified as 'forgetting' data still retain class structure deep in their representations, breaking the illusion of compliance.
Takeaways
- Mirage framework: New auditing tool from Fudan University reveals that vision models certified as "forgetting" data still retain class structure in internal representations. [Paper]
- Forgetting gap: Models passing all standard output-level unlearning tests still show LPR scores up to 15.4 points above the retrained baseline, exposing hidden memory.
- Unlearning trilemma: No existing method simultaneously achieves high utility, output-level forgetting, AND representation-level forgetting across 7 datasets and 7 baselines.
- Class-sample asymmetry: Class-level forgetting leaves LPR scores up to 97%; sample-level forgetting is indistinguishable from chance (~50% LPR), the most common GDPR use case.
- Four diagnostics: Linear Probe Recovery, CKA, Feature Separability Scoring, and Layer-Wise Recovery Analysis form a complete representation-level audit pipeline.
- Compliance risk: With EDPB making right-to-erasure its top enforcement priority, current VFL unlearning certifications may be legally insufficient.
Machine unlearning -- the ability to make a trained model forget specific data -- has become one of the most pressing problems in AI compliance. Recent legal requirements, such as the right to be forgotten, require that an unlearned model behave as if it has never processed the specified data. But a new paper from researchers at Fudan University asks a sharp question: what if the model only appears to forget?
Machine unlearning in Vertical Federated Learning (VFL) has attracted growing interest, yet existing methods certify forgetting solely using output-level metrics. A team of researchers has now built Mirage, a framework that looks past those surface-level checks and audits what is actually happening inside the model's representations -- and the results are damning for the current state of the field.
The compliance pressure behind this work
The machine unlearning problem is the collision point between GDPR's right to erasure and the fundamental way machine learning works. A user's data influenced the model weights, shaped the patterns the system learned, and affected predictions being made for other users. Deleting the database rows does not undo what the model learned from that data -- the influence lives on in the parameters.
This is especially thorny in Vertical Federated Learning. VFL operates by vertically partitioning data features and labels across multiple entities, allowing them to jointly train machine learning models while preserving data privacy. Think of a bank and a hospital jointly training a model: the bank holds financial features, the hospital holds medical records, and neither party shares raw data. Recent privacy regulations emphasize an individual's right to be forgotten, which necessitates the ability for models to unlearn specific training data -- and the primary challenge is to eliminate the influence of a specific client from a model without erasing all relevant data from other clients.