Qwen3.8-27B gets a 4-bit MLX build for Apple silicon

The mlx-community organization has released a 4-bit conversion of Alibaba’s Qwen3.8-27B, a dense multimodal model that accepts text, images, and video. Its MLX repository showed more than 800,000 downloads when checked, although Hugging Face counts can include automated pulls and do not represent unique users.

The conversion gives Apple silicon developers a practical route to local multimodal inference. The checkpoint can fit in 24 GB of unified memory for short prompts, while 32 GB leaves more room for the operating system, visual inputs, generated tokens, and the context cache.

One checkpoint, three modalities

Qwen describes the upstream checkpoint as a 27-billion-parameter dense model, meaning every token uses the full network rather than routing through a subset of experts. Its architecture combines linear-attention blocks with periodic full-attention layers, balancing long-context efficiency with conventional token-to-token attention.

• Depth: 64 transformer layers.
• Attention: 24 query heads and four key-value heads on the full-attention path, using grouped-query attention to reduce cache memory.
• Context: 262,000 native tokens, with YaRN scaling advertised for contexts beyond one million tokens.
• Vocabulary: Approximately 248,000 tokens.
• Inputs: Text, images, and video through an attached vision encoder.
• Generation: A multi-token-prediction head is included, although any decoding speed-up depends on runtime support.

The advertised context limit describes the architecture, not a promise that the full window will fit on a laptop. Key-value caches, image tokens, video frames, and generated output consume additional unified memory, so practical context lengths depend on the machine and workload.

Quantization changes the memory math

MLX affine quantization stores groups of weights at lower precision alongside scaling metadata. The 4-bit group-64 conversion reduces the roughly 55.6 GB BF16 checkpoint to about 16 GB, with reported runtime use reaching approximately 16 to 19 GB before large contexts or visual inputs add memory pressure.