LocalAI's parakeet.cpp Runs NVIDIA's Parakeet 27x Faster Without Python
parakeet.cpp brings NVIDIA's best speech-to-text models to any CPU or GPU, with no Python, no PyTorch, and output identical to the original NeMo framework
Takeaways
- parakeet.cpp is a C++17/ggml port of all NVIDIA Parakeet ASR models, requiring no Python, PyTorch, or CUDA toolkit at inference time.
- Output is byte-for-byte identical to NeMo on all 10 published checkpoints (WER 0), while being up to 1.86x faster on CPU and up to 4.3x faster on GPU.
- All models are available as GGUF files on Hugging Face, from a 131 MB Q4_K 110M model up to 2.4 GB F16 1.1B variants.
- Supports CUDA, Metal, Vulkan, and HIP GPU backends, plus cache-aware streaming with real-time end-of-utterance detection.
- Ships as a flat C API embeddable from C, C++, Go, or Rust, and as a LocalAI backend with an OpenAI-compatible transcription endpoint.
- Benchmarks show roughly 27x faster than whisper.cpp on CPU at comparable accuracy for English speech.
NVIDIA's Parakeet family has quietly become one of the best open speech-to-text models available, but running it has always required dragging in the full NeMo framework: Python, PyTorch, CUDA toolkit, and a pile of dependencies. The parakeet.cpp project, built by the team behind LocalAI, just changed that. It is a from-scratch C++17 port of every Parakeet model, built on ggml (the same tensor engine that powers llama.cpp and whisper.cpp), and the GGUF weights are now available on Hugging Face.
The Python tax on speech recognition
NeMo is a great training framework, but running Parakeet just for inference drags in a heavy Python/PyTorch stack. For production deployments, edge devices, or any environment where you want a single binary you can ship, that overhead is a real cost. parakeet.cpp is a C++17 inference port of NVIDIA's NeMo Parakeet speech-recognition models, built on ggml. It gives you fast, dependency-light automatic speech recognition on CPU and on GPU through ggml's backends, with no Python runtime needed at inference time.
The result is a single libparakeet.so (or static lib) behind a flat C API, easy to embed from C, C++, Go, or Rust. It also ships as a backend inside LocalAI, giving you an OpenAI-compatible /v1/audio/transcriptions endpoint that runs fully locally.
Every model, every quantization
It covers all the offline Parakeet families: CTC, RNNT, TDT, and hybrid TDT-CTC, in 0.6B/1.1B/110M sizes, English plus multilingual v3, each validated at WER 0 against NeMo on every published checkpoint. The GGUF repo collects every model and quantization level in one place. Here is what is available:
- tdt_ctc-110m: the 267 MB F16 file is the fastest option for lightweight transcription
- realtime_eou_120m-v1: cache-aware streaming model with end-of-utterance detection, ideal for live applications