Slotstream Runs a 125B Open-Source Model on a Mac SSD

Slotstream is an open-source Swift inference engine that runs a 125-billion-parameter mixture-of-experts model on Apple Silicon Macs with 16 to 64 GB of unified memory. It keeps the 105 GB model on an SSD, loading the required expert weights into memory as it generates each token.

The approach makes a large local model available on Macs that lack enough memory to hold every weight at once. It supports offline coding agents, private chat, image questions, and other workloads that benefit from local inference.

Turn the SSD into an expert cache

Mixture-of-experts models route each token through a small subset of specialized subnetworks called experts. Slotstream keeps shared weights in unified memory and reads the selected experts from SSD into a cache. Frequently used experts remain resident, while the cache adjusts when other applications need memory.

The supported model is Qwen3.8-Flash-Next, a 125B MoE model quantized to 4 bits. Running it requires:

• An Apple Silicon Mac with at least 16 GB of unified memory
• macOS 14 or later
• About 110 GB of free SSD space

An 8 GB Mac cannot accommodate the smallest memory plan, so the engine exits instead of forcing macOS to swap. Windows, Linux, and Intel Macs are unsupported.

Fast decoding, costly prefill