OpenAI's Whisper large-v3 Runs Locally Under 1GB Without Python

The model repository packages OpenAI’s 1.5-billion-parameter Whisper large-v3 as GGUF files for transcribe.cpp, a new ggml-based inference engine. The smallest file occupies 997 MB and runs locally across several hardware backends, giving developers a compact multilingual transcription option without a hosted API or Python runtime.

GGUF stores model tensors and metadata in a single file designed for efficient loading by ggml runtimes. Quantization reduces the precision of those tensors, shrinking storage and memory requirements at the cost of some numerical fidelity. The repository reports only a 0.05-percentage-point increase in word error rate between its F16 and Q4_K_M files.

At publication time, the repository had recorded more than 829,000 downloads and 3,100 likes on Hugging Face. Its model page lists the files under Apache 2.0, which permits commercial use subject to the license terms and notice requirements.

One model, five memory budgets

The conversion preserves large-v3’s core capabilities: transcription across roughly 100 languages, automatic language identification, segment timestamps, and speech translation into English. Compared with earlier Whisper checkpoints, large-v3 processes 128-bin log-Mel spectrograms instead of 80-bin inputs and includes a Cantonese language token.

The reference file stores weights in 16-bit floating point. Four quantized versions reduce the file size by representing most weights with fewer bits, with the K-series formats applying mixed precision to protect selected tensors.

Word error rate counts substitutions, deletions, and insertions against a reference transcript; lower values indicate fewer errors. LibriSpeech test-clean consists of clean English audiobook speech, so these results provide little evidence about noisy recordings, far-field microphones, accents, or multilingual accuracy.