CVPR 2026's OTC Beats Catastrophic Forgetting by Treating Classes as Moving Mixtures
A new continual learning method uses optimal transport to grow multiple adaptive cluster centers per class, cutting forgetting on streaming data.
PRO- CVPR 2026 paper introduces MMOT, an online mixture-of-Gaussians learned via optimal transport for class-incremental learning. Paper
- Each class gets multiple adaptive centroids that update every mini-batch, instead of one fixed prototype.
- Wasserstein distance replaces EM, using entropic dual form, reparameterization, and Gumbel-Softmax for gradient-based updates.
- A Dynamic Preservation loss and centroid-based replay buffer selection reduce catastrophic forgetting.
- Beats ER, CoPE, OCM, GSA, MOSE, BiC+AC on CIFAR-10, CIFAR-100, Tiny-ImageNet, MNIST across memory sizes.
- Best number of centroids depends on buffer size; more centroids need more replay samples to stay useful.
Training a model on a never-ending stream of new classes is one of the ugliest problems in machine learning. You do not get to loop over the data, you cannot store much of it, and each new mini-batch quietly rewrites the features the model learned yesterday. A paper accepted at CVPR 2026, An Optimal Transport-driven Approach for Cultivating Latent Space in Online Incremental Learning, proposes a fix that treats each class as a moving mixture of Gaussians and updates it with cheap gradient steps instead of the usual EM iterations.
The setting nobody likes: online class-incremental learning
Online Class Incremental Learning (OCIL) is the hardest flavor of continual learning. Data arrives as small batches, the class distribution shifts over time, the model is only allowed a single update per batch, and at inference time you do not know which task a sample came from. Most methods stash a small replay buffer and try to keep old classes recognizable, but they still fight catastrophic forgetting: previously learned features drift as the backbone chases new data.
The dominant trick has been to represent each class by a prototype in latent space and pull samples toward it. Some approaches use one adaptive centroid per class, others use several fixed centroids fit with a Gaussian Mixture Model. Both have known weaknesses. A single prototype cannot capture a class whose data splits into several visual modes, and fixed GMM centroids go stale because the latent space keeps shifting as the model adapts to new data. MMOT addresses this by flexibly updating centroids with gradient descent instead of EM, which would require multiple inner iterations per learning step.
Turning a mixture model into a transport problem
This story is for Pro members
You've reached the end of the free preview. Upgrade to AlphaSignal Pro to read the full article - and everything else behind the paywall.
Comments
