Skip to main content
Back

Cleave Trains 70B Models on Consumer Devices, Matching Cloud A100 Speed

Cleave turns idle phones and laptops into a distributed training cluster by exploiting a match between how GEMMs move data and how home networks work.

Pro

Paper, Full Access
  • Cleave introduces a parameter-server-centric framework for training foundation models on idle edge devices.
  • Exploits GEMM I/O asymmetry: large inputs go over downlink, small partial outputs return over uplink.
  • Achieves cloud-comparable per-batch runtime and beats prior edge training by 4 to 10 times.
  • Scales to 8,192 heterogeneous devices where DTFM and Alpa run out of memory or bandwidth.
  • Recovers from device failures at least 100 times faster by resharding sub-GEMMs across survivors.
  • Preserves exact gradient semantics; no code or model has been released yet.

A team from the University of Edinburgh, Johns Hopkins, and Cisco Research has built a training framework that claims to match cloud performance while running foundation-model training across thousands of consumer edge devices. The system, called Cleave, targets a specific mismatch between how existing distributed training moves data and how home internet connections actually work, and the resulting speedups over prior edge-training methods are large enough to make the approach worth a serious look.

Why edge training keeps failing

Training frontier models sits with a handful of cloud operators because the compute bill is enormous. Volunteer computing on phones and laptops sounds like an obvious alternative, given billions of idle devices sitting on chargers for hours each day, but every prior attempt hits the same wall.

The authors frame the problem along three axes. Typical usable memory on phones is around 512MB, while the memory demand of training a model can be hundreds of GB. Data and pipeline parallelism alone cannot squeeze training into that budget. Tensor parallelism can, but it introduces AllReduce and AlltoAll traffic at every layer, which is fine on NVLink and disastrous on residential broadband. Devices also disappear constantly: even with a conservative interruption rate of 1% per device per hour, the system-level mean time between failures drops to about 47 minutes at 128 devices and below 6 minutes at 1,024.

The asymmetry nobody was exploiting

Cleave's core insight is structural rather than algorithmic. Across the LLaMA family, GEMMs account for more than 99% of total training FLOPs for 7B, 13B, and 70B models, while layer normalization, activation functions, and softmax together contribute less than 1%. Optimizing training essentially means optimizing GEMM execution.

Every GEMM has an input-heavy, output-light profile. The two input matrices shipped to a worker are substantially larger than the partial output block that worker sends back. That maps almost perfectly onto how edge networks are provisioned: per-device uplink speeds are typically 2 to 10 times slower than downlink, with typical downlinks of 10 to 100 MB/s and uplinks of 5 to 10 MB/s. Send the big inputs down the fat pipe, collect the small outputs on the thin one.