Skip to main content
Back

AMC Slashes AI Forgetting by 80% Without Growing the Network

A new continual learning framework treats replay memories like matter that freezes over time, cutting forgetting by up to 80%.

Pro

Paper, Full Access
  • AMC introduces a Liquid-Glass-Crystal replay buffer where each experience has a crystallization state evolving under an Itô SDE.
  • Improvements: +34-43% forward transfer, 67-80% less forgetting, 62% smaller memory footprint vs strongest baselines.
  • Benchmarks: Meta-World MT50, sequential Atari-20, and MuJoCo continual locomotion, with 50-seed statistics.
  • Closed-form Beta stationary distribution and Q-learning error bounds link SDE parameters directly to agent accuracy.
  • Crystallization gates per-sample learning rate as (1-c)^2, protecting consolidated experiences without freezing weights.
  • Drop-in for off-policy agents (SAC, DQN, TD3) with roughly 15% wall-clock overhead vs PER.

Autonomous agents that keep learning after deployment have a nasty habit: the moment they pick up a new skill, they tend to blow away an old one. A new paper introduces Adaptive Memory Crystallization (AMC), a replay-buffer architecture that borrows a metaphor from physics and neuroscience to keep useful experiences from getting overwritten, while still letting new ones flow in freely.

The headline numbers are aggressive. Across Meta-World MT50, Atari-20 sequential learning, and MuJoCo continual locomotion, AMC delivers 34-43% improvements in forward transfer over the strongest baseline, 67-80% reductions in catastrophic forgetting, and a 62% cut in memory footprint. It does so without growing the network or adding new heads per task.

The stability-plasticity headache

Continual reinforcement learning agents have been stuck on the same dilemma for years. Standard deep RL agents rely on a fixed-size experience-replay buffer combined with stochastic gradient descent, and when the task distribution shifts, gradient updates on new data overwrite weights encoding old behaviors, a phenomenon known as catastrophic forgetting.

Existing fixes fall into three camps, each with a known failure mode:

  • Regularization (EWC, SI, MAS): penalize changes to weights important for old tasks. The constraint matrix grows with task count, which can severely limit plasticity at scale and leads to negligible forward transfer on long task sequences.
  • Dynamic architectures (Progressive Networks, PackNet): grow or partition capacity per task. Memory cost grows linearly in the number of tasks, which becomes impractical beyond moderate task counts for resource-constrained agents.
  • Replay methods (PER, HER, NEC): curate which experiences to replay. They provide no consolidation signal that progressively locks in experiences by long-term value.

Memory as a phase transition

AMC's central move is to treat each stored experience as a substance that can be liquid, glass, or crystal, with a continuous state variable governing which phase it belongs to. Each buffered experience carries a scalar crystallization state between 0 and 1, where 0 represents a fully plastic memory and 1 a fully stable one. That state evolves stochastically, driven by how useful the experience seems to be.