Skip to main content
Back

CMU's PubSwap Lets Hospitals Train Reasoning AI Without Sharing Private Data

A CMU team combines LoRA aggregation with shared public prompts to keep federated reasoning models coordinated without leaking private data.

  • CMU proposes PubSwap, a federated RLVR method combining LoRA aggregation with public-data off-policy steps.
  • Clients swap responses on shared public prompts rather than gradients, keeping private data local.
  • Balanced rule replaces only locally incorrect answers with globally correct ones to limit off-policy drift.
  • Beats FedAvg-GRPO and FedProx-GRPO on MATH, DeepMath, and medical reasoning across Qwen and Llama models.
  • Biggest gains appear at high local-step counts, the regime where communication savings matter most.
  • Built on the VERL framework using LoRA rank 32 and 4 simulated clients on H100s.

Training reasoning models with reinforcement learning from verifiable rewards (RLVR) has become the go-to recipe for pushing math, code, and medical reasoning benchmarks. Nearly all of that work assumes one organization sitting on one giant pile of data. Hospitals, banks, and law firms don't have that luxury, and a new paper from Carnegie Mellon proposes a practical way to run RLVR across siloed datasets without shipping either the data or the full model weights around.

The method, called PubSwap, combines LoRA adapters for cheap parameter exchange with a small shared public dataset that clients use to swap answers rather than data. By interleaving private-data GRPO steps with public-data off-policy steps, PubSwap improves cross-client alignment while preserving communication efficiency and privacy.

Why federated RLVR breaks the usual playbook

RLVR post-training is typically studied in centralized settings, yet many realistic applications involve decentralized private data distributed across organizations. Federated learning is the obvious fix, but two things break when you plug GRPO into the standard FedAvg loop. Full-model synchronization is expensive, and running many local steps between syncs causes severe client drift under heterogeneous data. Clients holding wildly different distributions, say cardiology notes versus oncology notes, end up pulling the shared model in incompatible directions.