Backburner turns an iPhone into a local LLM accelerator

Backburner is an open-source project that distributes local large language model inference across an Apple Silicon Mac and an iPhone. Connect the devices with a 10 Gb/s USB-C cable, launch the companion iOS app, and the phone contributes its GPU and Neural Engine to a 27-billion-parameter model running through llama.cpp.

In the project’s reported benchmarks, the added hardware increased long-context prompt processing throughput by as much as 44%. It also let a 24GB Mac extend an 8-bit key-value cache beyond its approximate 64,000-token memory limit, with tests reaching 128,000 tokens.

One cable, two compute paths

During prompt processing, also called prefill, Backburner divides the model by layer. The Mac processes layers 1 through 40 for each 256-token microbatch, then sends the residual activations, the intermediate values passed between model layers, to the phone. The phone runs layers 41 through 64 on its GPU while the Mac begins the next microbatch.

This pipeline keeps both processors working and limits cable traffic to intermediate activations. The model weights remain split across the devices, with about 5.1GB of upper-layer weights stored on the phone.

Once the context exceeds the Mac’s memory budget, Backburner also moves the oldest key-value cache pages to the phone. A KV cache stores the attention data from earlier tokens so the model does not recompute them for every generated token. Backburner organizes those entries into pages containing 4,096 keys each.

For each attention step, the Mac sends the query tensor to the phone. The phone calculates attention against its cached keys and returns partial output, maximum and sum values, which the Mac merges with its local result. A custom GPU matrix kernel performs this work.