NVIDIA's Parakeet v3 Now Runs Without Python or a GPU
A community port by the handy-computer team converts NVIDIA's multilingual Parakeet-TDT-0.6B-v3 into GGUF files for transcribe.cpp, enabling local CPU and GPU inference.
- The handy-computer team released GGUF conversions of NVIDIA's Parakeet-TDT-0.6B-v3 multilingual ASR model.
- Six quantizations from F32 (2.51 GB) to Q4_K_M (485 MB), all within 0.05% WER of the reference.
- Runs through transcribe.cpp, a C/C++ ggml-based runtime with Metal, Vulkan and CUDA backends.
- Covers 25 European languages with automatic language detection, punctuation, and word-level timestamps.
- Benchmark on Intel Iris Xe shows roughly 12.5x realtime via Vulkan, 6.8x via CPU.
- Offline only, no streaming or translation; license is CC-BY-4.0 inherited from NVIDIA.
NVIDIA’s Parakeet-TDT-0.6B-v3 speech-recognition model can now run through transcribe.cpp without Python, NeMo, or CUDA. The handy-computer team converted the checkpoint to GGUF, a format designed for efficient inference in native runtimes, and published six precision levels in a model repository.
The release is an independent port rather than an NVIDIA distribution. It gives C and C++ applications a simpler route to local transcription on CPUs, integrated GPUs, and supported discrete GPUs while retaining the upstream model’s 25-language coverage.
Six files, one checkpoint
The conversion targets NVIDIA’s checkpoint and records upstream commit 6d590f7. The maintainers validated the port against NVIDIA’s NeMo implementation.
Parakeet-TDT-0.6B-v3 contains about 600 million parameters. Its Conformer encoder analyzes audio, while its TDT transducer decoder emits text tokens and timing information. The model accepts 16 kHz mono audio, supports optional timestamps, detects the spoken language automatically, and produces punctuation and capitalization. This checkpoint handles offline transcription only; it does not stream or translate speech.
Quantization stores model parameters at lower precision, reducing file size and memory use. The repository provides these variants, all benchmarked on LibriSpeech test-clean. Lower word error rate, or WER, indicates fewer transcription errors.
| Variant | File size | WER |
|---|---|---|
F32 |
2.51 GB | 1.95% |
F16 |
1.26 GB | 1.95% |
Q8_0 |
740 MB | 1.94% |
Q6_K |
610 MB | 1.93% |
Q5_K_M |
549 MB | 1.92% |
Q4_K_M |
485 MB | 1.98% |
The six results span 0.06 percentage points, while the difference between F32 and Q4_K_M is 0.03 points. Small reversals among the variants likely reflect benchmark variation. NVIDIA reports 1.93% WER on the same split, placing the conversions close to the reference implementation.
This story is for Pro members
You've reached the end of the free preview. Upgrade to AlphaSignal Pro to read the full article - and everything else behind the paywall.
Comments
