University of Florence's SOLAR Stops AI from Forgetting Unlabeled Data Streams
New research shows replay buffers silently poison self-supervised models on streaming data, and proposes SOLAR to adaptively balance stability and plasticity.
ProTakeaways
- New paper identifies Latent Rehearsal Decay, a collapse phenomenon in online continual self-supervised learning caused by over-stable replay buffers.
- Counterintuitive finding: a simple FIFO buffer beats Reservoir sampling in OCSSL because Reservoir over-rehearses the same samples.
- Two new diagnostic metrics, Overlap and Deviation, correlate with drops in linear probing accuracy.
- SOLAR combines a deviation-aware buffer, extraction-count-weighted sampling, and an explicit overlap loss.
- Achieves state-of-the-art on Split CIFAR-100 and Split ImageNet100 with ResNet-18, especially at small buffer sizes.
- Paper on OpenReview; no public code release announced yet.
Replay buffers are the workhorse of continual learning: keep a small sample of past data around, mix it into every training step, and forgetting mostly goes away. A new paper argues that this recipe quietly breaks down in one important setting, and it introduces both a diagnosis and a fix.
The setting is Online Continual Self-Supervised Learning, or OCSSL, where a model learns from a continuous stream of unlabeled data. Here, a simple FIFO buffer outperforms Reservoir sampling, which is counterintuitive. In standard continual learning, Reservoir sampling, which keeps an unbiased sample of the stream, is the default choice. In OCSSL, the naive first-in-first-out queue often wins.
The authors, from the University of Florence and the University of Pisa, call the culprit Latent Rehearsal Decay, and they build a method called SOLAR (Self-supervised Online Latent-Aware Replay) around preventing it.

Why replay quietly rots your latent space
To understand the problem, it helps to be precise about the scenario. In online continual learning, data arrives as a one-pass stream of small minibatches, which precludes multi-epoch training. Combine that with self-supervised learning, which normally craves huge batches, and you have a hard problem: small streaming minibatches are insufficient for effective learning, especially in SSL for which large batches are crucial. Replay buffers are used to bulk up each step with additional samples.
Here the classical stability-plasticity trade-off kicks in. Reservoir keeps a broad, representative snapshot of everything seen so far (stable). FIFO throws out the oldest items to make room for the newest (plastic). Stability should mean less forgetting. Instead, the authors find the opposite in OCSSL: latent space degradation happens under excessive stability of replay. Rehearsing the same handful of old images too many times, using ever-shifting self-supervised signals, corrupts the representation itself.