Cohere Transcribe Runs Offline on a Laptop With No Python Needed
A community port of Cohere Transcribe now runs offline through a llama.cpp-style C++ runtime, with four quantizations that hold accuracy within rounding.
Takeaways
- Community developer handy-computer released GGUF conversions of Cohere Transcribe for the transcribe.cpp runtime.
- Six quantizations from BF16 (4.10 GB) down to Q4_K_M (1.55 GB) are available under Apache 2.0.
- WER on LibriSpeech test-clean stays between 1.25% and 1.27% across all quantizations.
- Base model is Cohere Labs' 2B Conformer encoder-decoder ASR supporting 14 languages.
- Cohere Transcribe currently leads the Open ASR Leaderboard with 5.42 average WER.
- Limitations: no language detection, no timestamps, no speaker diarization, needs VAD upfront.
An independent developer going by handy-computer has published GGUF conversions of Cohere Labs' open-source speech recognition model, making it possible to run the 2B-parameter transcriber offline through a lightweight C++ runtime instead of the Python and PyTorch stack. The port targets transcribe.cpp, a project modeled on the llama.cpp pattern but built for automatic speech recognition, and it ships with six quantizations ranging from BF16 down to Q4_K_M.
The underlying weights come from Cohere Transcribe, a 2B-parameter model Cohere Labs released on Hugging Face under an Apache 2.0 licence. The architecture is conventional and well-tested, with most of the development cycles spent on data work. The GGUF repo repackages those weights without retraining or architectural changes, and inherits the upstream license.
Small quants, same word error rate
The interesting technical claim is that the smaller quants barely move the needle on word error rate. Measured on the full LibriSpeech test-clean split with greedy decoding and no external language model, the numbers land like this:
| Quantization | Size | WER (test-clean) |
|---|