Edge0 Demos a 35B MoE on an iPhone

Edge0’s release demo shows a 35-billion-parameter language model running on an iPhone with a reported 2.9 GiB peak active-memory footprint. The Edge0 repository, published under Apache 2.0, includes a streaming Mixture-of-Experts inference framework and two checkpoints: a 35B model derived from Qwen3.5-MoE and an 8B model based on Ling 3.0.

The reported memory figure describes the model’s working set at short context lengths. The full 4-bit 35B checkpoint still occupies about 23 GB on disk. Edge0 reduces active memory by loading selected expert weights from local storage as each token moves through the model.

A 23 GB Model, 2.9 GiB at Work

Sparse Mixture-of-Experts models divide portions of the network into expert subnetworks, then use a router to select a small group for each token. Many runtimes keep every expert in memory to make that selection fast. Edge0 memory-maps the expert files, which lets the operating system load individual storage pages only when the model accesses them.

The resulting working set tracks the selected experts instead of the model’s total parameter count. Edge0 reports peak active memory of about 2.9 GiB for the 35B checkpoint and 1.0 GiB for the 8B checkpoint at short contexts. Frequently selected experts remain cached, while less active experts stay on storage until requested.

Storage reads introduce latency, particularly when the operating-system cache is cold. Edge0 overlaps those reads with model computation and uses quantization-recovery adapters to preserve more of the original model’s accuracy.

Latency Hiding in Three Parts

• Expert offload: Expert weights are memory-mapped on local storage and fetched on demand. Cached experts remain available for reuse without loading the entire checkpoint into memory.
• Prerouter prediction: A small trained head predicts the experts required by the next token. The runtime can begin reading those weights while it processes the current token. Edge0 reports up to 59 percent higher decode throughput with this prediction path.