---
title: "FlashbackCL Cuts Federated Learning's Forgetting Problem by 68%"
subtitle: "FlashbackCL fixes a blind spot in federated learning: what happens when the data on each device changes over time, not just across devices"
url: "https://alphasignal.ai/news/flashbackcl-cuts-federated-learning-s-forgetting-problem-by-68"
author: "AlphaSignal Newsroom"
author_url: "https://alphasignal.ai/editorial-team"
company: "AlphaSignal"
company_url: "https://alphasignal.ai/company/alphasignal"
published: "2026-06-02T00:00:00.000Z"
updated: "2026-06-10T15:56:56.404Z"
type: "Paper"
topics:
  - "Training Infra"
  - "Data"
subtopics:
  - "Distributed Training"
  - "Data Curation"
  - "Synthetic Data"
upvotes: 2811
source_url: "https://arxiv.org/abs/2606.03939"
comments: 0
truncated: true
---

# FlashbackCL Cuts Federated Learning's Forgetting Problem by 68%

> FlashbackCL fixes a blind spot in federated learning: what happens when the data on each device changes over time, not just across devices

Federated Learning (FL) -- training a shared model across many devices without ever centralizing their data -- has a well-known enemy: heterogeneity. Different phones, hospitals, or sensors hold very different data, and that mismatch causes the global model to forget what it learned from one group of devices as it absorbs updates from another. The [Flashback algorithm](https://arxiv.org/abs/2402.05558) became the strongest recent answer to this "spatial" forgetting problem, using per-class label counts as a proxy for how much knowledge each client holds. But Flashback was built on a silent assumption -- that each client's data distribution stays fixed over time. In the real world, it almost never does.

A new paper, [FlashbackCL](https://arxiv.org/abs/2606.03939), tears open that assumption and proposes a drop-in fix. The result is a system that handles both the classic cross-client heterogeneity problem and a newly formalized one: **temporal forgetting**, where the global model gets anchored to an outdated picture of the world as clients' data drifts through seasons, concept changes, or shifting user behavior.

## Two kinds of forgetting, one problem

To understand why this matters, it helps to know what Flashback actually does. Knowledge loss in FL occurs in both client-local updates and server-side aggregation steps; addressing one without the other fails to mitigate forgetting. Flashback's solution is dynamic knowledge distillation -- it uses per-class label counts accumulated across clients as a proxy for knowledge, then adapts the distillation loss accordingly. Across different benchmarks, Flashback outperforms other methods, mitigates forgetting, and achieves faster round-to-target-accuracy, converging in 6 to 16 rounds. 

The catch: those label counts are monotonically accumulating. They only go up. If a client's data distribution shifts -- say, a hospital's patient population changes, or a phone's user changes their habits -- the counts become stale. The model gets regularized toward an old class balance that no longer reflects reality, anchoring the global model to the past.

_Preview only. Full text requires an AlphaSignal Pro account._

## Sources

- [FlashbackCL: Mitigating Temporal Forgetting in Federated Learning](https://arxiv.org/abs/2606.03939)
- [Flashback: Understanding and Mitigating Forgetting in Federated Learning](https://arxiv.org/abs/2402.05558)
- [Federated Learning Under Evolving Distribution Shifts](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12839774/)
- [FedTeddi: Temporal Drift and Divergence Aware Scheduling for Federated Edge Learning](https://arxiv.org/pdf/2509.07342)
- [Rethinking Experience Replay: a Bag of Tricks for Continual Learning](https://arxiv.org/pdf/2010.05595)