Liquid AI's LiquidCodec Encodes Speech 361x Faster for On-Device AI
Liquid AI's 74M-parameter streaming audio codec encodes at 361x real time on a single CPU thread, beating Mimi, X-Codec 2, and NanoCodec on speech generation quality.
- Liquid AI released LiquidCodec, a ~74M-param streaming audio codec for LFM3-Audio-2B and LFM3-Omni-2B.
- Runs ~361x real-time encode, ~87x decode on a single M4 Max CPU thread at FP32.
- Uses grouped finite scalar quantization instead of RVQ, enabling parallel multi-codebook token prediction.
- Best ViSQOL and mel distance on all 8 test sets; top TTS naturalness across the board.
- Trained in two stages: 264k hours across 107 languages, then 55k hours adversarial fine-tune.
- Ships Q4_0, Q4_K_M, Q8_0 GGUF builds for llama.cpp on-device inference.
Liquid AI has released LiquidCodec, an approximately 74-million-parameter streaming audio codec used by its LFM3-Audio-2B and LFM3-Omni-2B models. Its grouped token design targets faster codec processing and simpler speech-token prediction on local hardware.
In a speech-language model, a neural codec converts waveform samples into discrete token IDs and reconstructs audio from predicted IDs. Reconstruction quality affects output fidelity, while the token structure determines how much work the language model must perform to generate speech.
Why Liquid moved beyond Mimi
Liquid previously paired LFM2-Audio-1.5B with Mimi, a roughly 96-million-parameter codec that operates at 1.1 kbps. For LFM2.5-Audio-1.5B, the company replaced Mimi’s detokenizer and reported an eightfold decoding speedup, while retaining Mimi’s tokenizer and its residual vector quantization design.
Residual vector quantization, or RVQ, processes a sequence of codebooks. Each codebook encodes the error left by the preceding one, creating dependencies between their outputs. Speech models must account for that ordering when predicting codec tokens, which limits opportunities for parallel generation.
Independent groups unlock parallel prediction
LiquidCodec uses grouped finite scalar quantization, or GFSQ. It divides each feature vector into groups, quantizes each group independently, and maps the resulting value combination to a token. The groups can then be predicted together through MaskGIT-style refinement, which fills masked tokens over successive passes.
A lean 24 kHz path
- Fixed frame rate: The codec emits 12.5 frames per second, giving each frame an 80 ms span.
- 24 kHz audio: The sample rate balances speech bandwidth with compute and memory demands.
- Causal streaming: The encoder uses LFM3 blocks and processes current and past audio without waiting for future samples.
- Direct spectral output: The decoder predicts spectral coefficients directly, avoiding the progressive learned upsampling used by many neural codecs.
Broad pretraining, cleaner finish
Liquid trained the codec in two data stages, followed by a decoder-only pass for low-precision deployment.
| Stage | Data | Method | Result |
|---|---|---|---|
| Multilingual pretraining | 264,000 hours across 107 languages | Waveform and spectral reconstruction losses applied to web speech, audiobooks, crowdsourced recordings, meetings, and parliamentary audio | Broad language coverage, with a metallic artifact remaining in reconstructed speech |
| Full-band fine-tuning | 55,000 hours of cleaner audio recorded at 24 kHz or higher | Adversarial critics evaluated waveform patterns, spectral detail, and learned speech features | Liquid reports that the additional perceptual training removed the metallic artifact |
| Deployment pass | Decoder only | Quantization-aware training | Preparation for low-precision edge inference |
Liquid attributes the first-stage artifact to reconstruction objectives averaging away fine perceptual detail. The adversarial critics added in stage two rewarded outputs that more closely matched the structure of real speech.
Eight test sets reveal the trade-offs
According to Liquid, its controlled comparison trained each codec on the same 101,000 hours across six languages and held the TTS architecture and compute budget constant. Each codec was paired with an LFM3-1.7B backbone and a six-layer autoregressive depth decoder, then evaluated on eight LibriSpeech, Seed-TTS, and FLEURS test sets.
| Area | Metric | Reported result |
|---|---|---|
| Reconstruction quality | ViSQOL, where higher is better | Highest score on all eight test sets |
| Spectral accuracy | Mel-spectrogram distance, where lower is better | Lowest distance on all eight test sets |
| TTS naturalness | UTMOSv2, a model-based estimate of perceived quality | Highest score on all eight test sets |
| Intelligibility | Word and character error rates | Best result on four of eight test sets |
| Quantized decoding | Q4_0 decoder weights | 3.5-fold smaller decoder with limited reported quality loss |
| Backbone sensitivity | Qwen3-1.7B substituted for LFM3-1.7B | Relative codec gains persisted after the backbone change |
CPU throughput leads the results
Liquid measured FP32 performance on one CPU thread of a MacBook Pro with an Apple M4 Max. Under that configuration, the codec processed audio substantially faster than real time.
| Task | LiquidCodec | Reported comparison |
|---|---|---|
| Encoding | Approximately 361× real time | About 13× faster than Mimi and more than 250× faster than X-Codec 2 |
| Decoding | Approximately 87× real time | About 3× faster than Mimi and 22× faster than NanoCodec |
A rate of 361× real time means the encoder can process about 361 seconds of audio per wall-clock second under the stated setup. End-to-end response time also depends on input buffering, the language model’s token generation rate, codec frame cadence, and audio playback. The published throughput figures cover codec execution rather than the complete speech pipeline.
A codec swap changes the vocabulary
- Vendor-run evaluation: Liquid produced the published benchmarks. Teams should reproduce them on target hardware, languages, speakers, and acoustic conditions.
- Mixed secondary metrics: NanoCodec led several non-English intelligibility tests and scored better on some PESQ and speaker-similarity measurements.
- Generation-focused trade-offs: Pipelines centered on waveform reconstruction may rank codecs differently from TTS systems that need predictable language-model tokens.
- Token incompatibility: A speech model trained on Mimi, Encodec, or SoundStream tokens cannot directly consume LiquidCodec tokens. Replacing the codec requires model retraining or adaptation.
The shortest deployment path uses LFM3
The standalone codec is available on Hugging Face and loads through Transformers. Liquid also provides two speech models with the codec already integrated:
- LFM3-Audio-2B for audio-focused generation
- LFM3-Omni-2B for multimodal workloads
Both GGUF repositories include Q4_0, Q4_K_M, and Q8_0 builds for llama.cpp inference. The reported 3.5-fold reduction applies specifically to the codec decoder under Q4_0 quantization, so full-system memory and speed should be measured separately.
New local TTS and voice-assistant projects can use the integrated LFM3 models without training a speech model around a new token vocabulary. Existing systems need an adaptation plan and workload-specific tests covering intelligibility, speaker retention, long-form stability, time to first audio, and sustained CPU use. LiquidCodec’s clearest reported advantage lies in generation-heavy edge systems where codec throughput and token predictability share priority with reconstruction quality.
Comments
