FlashbackCL Cuts Federated Learning's Forgetting Problem by 68%
FlashbackCL fixes a blind spot in federated learning: what happens when the data on each device changes over time, not just across devices
- FlashbackCL extends the Flashback FL algorithm to handle temporal distribution shift, not just cross-client heterogeneity.
- The core problem: Flashback's monotonically accumulating label counts become stale when client data drifts over time, anchoring the model to an outdated class balance.
- Three fixes are introduced: temporally-decayed label counts, a device-aware CBRS replay buffer, and server-side active coreset curation.
- Results on CIFAR-10 (50 clients, 3 shift modes) show 6.9-10.0% relative accuracy gain over Flashback and up to 68% reduction in temporal forgetting.
- CBRS replay is identified as the single most critical component via a 5-variant ablation study.
- FlashbackCL also improves Flashback by 3.5 points on stationary CIFAR-100, suggesting replay regularizes spatial heterogeneity too; no code released yet.
Federated Learning (FL) -- training a shared model across many devices without ever centralizing their data -- has a well-known enemy: heterogeneity. Different phones, hospitals, or sensors hold very different data, and that mismatch causes the global model to forget what it learned from one group of devices as it absorbs updates from another. The Flashback algorithm became the strongest recent answer to this "spatial" forgetting problem, using per-class label counts as a proxy for how much knowledge each client holds. But Flashback was built on a silent assumption -- that each client's data distribution stays fixed over time. In the real world, it almost never does.
A new paper, FlashbackCL, tears open that assumption and proposes a drop-in fix. The result is a system that handles both the classic cross-client heterogeneity problem and a newly formalized one: temporal forgetting, where the global model gets anchored to an outdated picture of the world as clients' data drifts through seasons, concept changes, or shifting user behavior.
Two kinds of forgetting, one problem
To understand why this matters, it helps to know what Flashback actually does. Knowledge loss in FL occurs in both client-local updates and server-side aggregation steps; addressing one without the other fails to mitigate forgetting. Flashback's solution is dynamic knowledge distillation -- it uses per-class label counts accumulated across clients as a proxy for knowledge, then adapts the distillation loss accordingly. Across different benchmarks, Flashback outperforms other methods, mitigates forgetting, and achieves faster round-to-target-accuracy, converging in 6 to 16 rounds.
The catch: those label counts are monotonically accumulating. They only go up. If a client's data distribution shifts -- say, a hospital's patient population changes, or a phone's user changes their habits -- the counts become stale. The model gets regularized toward an old class balance that no longer reflects reality, anchoring the global model to the past.
This story is for Pro members
You've reached the end of the free preview. Upgrade to AlphaSignal Pro to read the full article - and everything else behind the paywall.
Comments
