Moondream Shrinks Parakeet Redux Speech Recognition to 178 MB
Moondream shrank NVIDIA's Parakeet speech model from 1.2GB to 178MB using ternary weights, hitting 113x realtime on CPU while matching accuracy.
- Moondream released Parakeet Redux, a ternary-quantized version of NVIDIA's Parakeet ASR model.
- Encoder weights are compressed to -1, 0, +1, shrinking the model from 1.2GB to 178MB.
- Runs at 113x realtime on 8 x86 CPU cores, 2.5x faster than parakeet.cpp.
- English WER rises only 0.29 points; beats the original on FLEURS 25-language and TED-LIUM long-form.
- Ships with built-in VAD for automatic chunking, plus streaming and live-mic APIs via Photon.
- Weakness is noisy audio; the Parakeet Ultra GPU variant is recommended when SNR is low.
Parakeet Redux compresses speech recognition to 178 MB
Moondream has released Parakeet Redux, a 178 MB speech-to-text model derived from NVIDIA’s 1.2 GB parakeet-tdt-0.6b-v3. It retains the original architecture, tokenizer, and support for 25 languages while constraining every encoder weight to -1, 0, or +1. The smaller representation enables fast local transcription on CPUs and Apple Silicon, with modest accuracy changes on clean speech and a larger regression in noisy conditions.
| Model size | 178 MB, down from 1.2 GB |
|---|---|
| Encoder weights | Three values: -1, 0, and +1 |
| Architecture | Same as parakeet-tdt-0.6b-v3 |
| Languages | English and 24 other European languages |
| License | CC BY 4.0 |
Ternary weights cut memory traffic
Local inference on CPUs and Apple Silicon often depends on memory bandwidth because the processor must repeatedly fetch model weights. Packing each encoder weight into one of three possible states reduces that traffic substantially compared with 8-bit or 16-bit representations. The “1.58-bit” label comes from log2(3), the information needed to represent three states before storage overhead.
Moondream’s Photon inference engine operates directly on the packed weights through hardware-specific kernels. It uses AVX-512 VNNI on supported x86 processors, NEON on ARM CPUs, and Metal on Apple GPUs.
This story is for Pro members
You've reached the end of the free preview. Upgrade to AlphaSignal Pro to read the full article - and everything else behind the paywall.
Comments
