Edge0 Streams an 8B AI Model From SSD Using Only 1 GiB
Edge0 releases an 8B sparse MoE that runs in under 1 GiB of active memory on Apple Silicon, streaming experts from SSD on demand.
- Edge0 released Edge0-8B-A1B-preview, an 8B sparse MoE running in ~1 GiB active memory at 25 tok/s.
- Built on Ling 3.0 tiny with 128 experts, K=8 routing, 4-bit quantization, 128k context, Apache 2.0.
- Ships with LoRA and prerouter adapters bundled; loads automatically via the edge0 CLI.
- Uses SSD expert offload plus one-step-ahead route prediction for up to +59% decode throughput.
- Average benchmark loss vs fp16 base is 2.8 points; MMLU-Pro actually beats the base.
- MLX/Apple Silicon only today; not yet tuned for agentic tool use or long horizons.
Edge0 previews an SSD-streamed 8B MoE with a 1 GiB active set
Edge0 has released the 8B checkpoint and an accompanying streaming runtime that load mixture-of-experts weights from SSD as each token needs them. On the project’s Mac mini benchmark, the 4-bit model decoded at 23.9 to 25.3 tokens per second while using 1 GiB of peak active memory.
The memory figure covers the active model working set. A deployment must also accommodate the 4.55 GB checkpoint on disk, the KV cache used for conversation context, the runtime, and other process memory. The current implementation targets Apple Silicon through MLX, so the release demonstrates phone-class active memory without providing a mobile runtime.
The preview derives from inclusionAI’s Ling 3.0 tiny base and ships under Apache 2.0. Its repository includes the 4-bit weights, Recover-LoRA adapters, prerouter adapters, and configuration required by the Edge0 CLI. Edge0 also released a larger 35B sibling.
An 8B checkpoint, 1.2B parameters at a time
A mixture-of-experts model contains many specialized feed-forward blocks, called experts, while a router selects a subset for each token. Edge0 has about 7.9 billion total parameters, but each token activates roughly 1.2 billion of them.
| Component | Specification |
|---|---|
| Layers | 24 |
| Hidden size | 1,536 |
| Experts | 128 per layer |
| Active experts | 8 per token, or K=8 |
| Parameters | Approximately 7.9B total and 1.2B active |
| Context window | Up to 128K tokens |
| Format | MLX 4-bit with LoRA and prerouter adapters |
| Disk size | Approximately 4.55 GB |
The chat template exposes a thinking mode. The repository also arrives as a ready-to-run directory for the Edge0 CLI, avoiding a separate conversion step.
SSD streaming makes the memory math work
Edge0 combines three mechanisms to keep most weights out of memory while limiting storage stalls:
This story is for Pro members
You've reached the end of the free preview. Upgrade to AlphaSignal Pro to read the full article - and everything else behind the paywall.
Comments
