Skip to main content
Back

NUS Researchers' HAMU Tells AI Models Exactly When to Stop Forgetting

HAMU quantifies how hard it is to forget training data and uses that signal to guarantee unlearning without unnecessarily destroying model utility

Paper, Full Access
  • New algorithm: HAMU introduces a hardness measure for machine unlearning that quantifies how similar forget and retain data are.
  • Core insight: Forget-retain similarity determines how hard unlearning is; HAMU uses this to switch between direct and rectified gradient updates.
  • Guaranteed tradeoffs: Unlike weighted-loss baselines, HAMU provides formal guarantees on forget quality improvement while bounding retain utility degradation.
  • Stopping criterion: HAMU tells users when both objectives cannot be improved simultaneously, preventing silent model degradation.
  • Broad applicability: Works on non-convex models, is parallelizable, and outperforms baselines (FT, GA, GDiff, KL, SCRUB) on both image and text benchmarks.
  • Open source: Two variants (HAMU-Q and HAMU-U) are available at github.com/aoi3142/HAMU.

Machine unlearning -- the art of making a trained model forget specific data without retraining from scratch -- has become one of the most practically urgent problems in AI. Driven by heightened concerns over user privacy, data quality, and the energy expenditure of training massive models like LLMs, the field has attracted significant attention. Regulations like the GDPR and the California Consumer Privacy Act (CCPA) establish legal obligations for the removal of personal data from automated systems, and retraining models from scratch is computationally prohibitive, making full deletion of internal representations difficult to verify. A new paper from researchers at A*STAR and the National University of Singapore takes a fresh angle on the problem -- and the key insight is surprisingly intuitive: how hard it is to forget something depends on how similar that something is to everything else the model needs to remember.

The tug-of-war at the heart of unlearning

Every unlearning algorithm faces the same fundamental tension. You have a forget set (the data you want erased) and a retain set (everything else the model must still perform well on). Machine unlearning naturally poses a multi-objective challenge: removing the influence of the forget set while maintaining overall performance. Most existing methods handle this by combining the two objectives into a single weighted loss -- essentially saying "forget this, but not too aggressively."

The problem is that this approach has no guarantees. If the forget set and retain set overlap statistically or semantically, blindly reducing the forget loss can catastrophically increase the retain loss. Balancing this requires careful tuning of a weighting hyperparameter for each instance, and even a dynamic approach provides no explicit control over the degradation on the retention set. Meanwhile, gradient ascent-based unconstrained optimization methods frequently fail to perform machine unlearning due to the inherent statistical dependence between the forget and retain datasets -- a dependence that undermines the assumption that these sets can be independently manipulated.