EPFL's DeepSet-TM Beats Poisoned AI Predictions by 22 Accuracy Points
EPFL researchers show federated inference is dangerously exposed to poisoned predictions, and their DeepSet-TM defense beats prior methods by up to 22 accuracy points.
Takeaways
- EPFL team formalizes robust federated inference and shows plain averaging is easily broken by corrupted clients.
- DeepSet-TM combines a permutation-invariant neural aggregator with coordinate-wise trimmed mean at inference.
- Adversarial training over probit-space perturbations replaces expensive permutation enumeration with subset sampling.
- Beats prior robust aggregators by 4.7 to 22.2 accuracy points on CIFAR-10, CIFAR-100, and AG-News.
- Introduces SIA attack that flips predictions via the second-most-probable class, breaking existing defenses like COPUR.
- Code released at sacs-epfl/robust-federated-inference; targets cross-silo settings with tens of clients.
Ensemble methods that combine predictions from multiple client-held models are quietly becoming a workhorse for privacy-preserving AI. One-shot federated learning, edge ensembles, and multi-LLM routers all rely on the same trick: keep the models local, query them, and let a central server aggregate the responses. A new paper from EPFL, the University of Copenhagen, and Sorbonne argues that almost none of this machinery has been stress-tested against a compromised client, and shows that a handful of poisoned predictions can flip the final answer with embarrassing ease.
The team formalizes the setting as robust federated inference and proposes DeepSet-TM, a permutation-invariant neural aggregator trained adversarially and paired with robust averaging at test time. Their composition beats existing robust aggregation methods by 4.7 to 22.2 accuracy points across diverse benchmarks.
Why aggregation is the weak link
In federated inference, clients retain proprietary local models while a central server queries them for predictions. Those predictions are then combined into a final answer, either through averaging-based rules or server-side aggregator neural networks. Model weights never leave the client, which is appealing, but it opens an obvious attack surface: any client can lie about its probit vector.
Robustness to model failures and poisoned outputs has been largely overlooked in this line of work, despite preliminary study. That gap matters because failures and errors are practically unavoidable, and undefended models fall to relatively simple attacks. Prior defenses like COPUR try to purify responses with an autoencoder, but they operate in logit space and collapse under stronger attacks.