How a 177B Qwen model runs from an 85 GB file on a 64 GB Mac

Atomic Chat has published GGUF quantizations of Qwen3.8-Flash-Next that use an SSD-backed lookup table to reduce memory pressure. The repository reports 36 generated tokens per second for the 85 GB quant on an M5 Max with 64 GB of unified memory, with the model’s vision components loaded.

The base model has 177 billion parameters and previews the architecture behind Qwen4, according to its model card. Its unusual parameter layout makes the result possible: a 39 GB n-gram table receives only a few kilobytes of logical reads per token, allowing it to remain memory-mapped on the SSD while the compute-heavy tensors stay in unified memory.

How 85 GB fits in 64 GB

The 177B model includes a 51B-parameter n-gram embedding table. For each token, the model hashes the recent three-token sequence and retrieves 16 rows containing 160 values each. That lookup touches about 2.7 KB of logical data per forward pass.

At 36 generated tokens per second, the requested values represent little bandwidth. Storage pages are larger than the logical payload, so Atomic Chat estimates actual random-read traffic at roughly 3 MB/s. The repository reports sub-100-microsecond NVMe access within a token-generation budget of about 28 milliseconds.

The design succeeds because the table’s read ratio is approximately one parameter in 13 million per token. Active mixture-of-experts layers touch about 6B parameters and move gigabytes of data, which still requires high-bandwidth memory. SSD-backed execution therefore applies to the sparse lookup bank, while the active model core remains resident in unified memory.

Apple Silicon adds a critical shard-layout constraint. When a GGUF shard contains GPU tensors, llama.cpp exposes the shard’s entire memory-mapped region to Metal. Interleaving the n-gram table with GPU weights can wire the table into memory and trigger kIOGPUCommandBufferCallbackErrorOutOfMemory during the first decode. Atomic Chat isolates the table in shard 2 for every quantization in the repository, so users should preserve the supplied shard layout.

Inside Qwen4’s preview design