Liquid AI's LFM2.5 Hits 97% Full-Precision Quality in a Tiny 4-Bit File

Shrinking a language model down to 4 bits usually means eating a noticeable quality hit. Liquid AI is now shipping a set of on-device checkpoints that mostly close that gap, and they still run on the same llama.cpp path everyone already uses.

Liquid AI released updated 4-bit checkpoints for LFM2.5-230M, LFM2.5-350M, LFM2.5-1.2B-Instruct, and LFM2.5-2.6B trained with Quantization-Aware Distillation (QAD). These checkpoints keep the low memory footprint and high throughput of Q4_0 GGUFs while recovering most of the accuracy lost to quantization: all four land at roughly 97% of their BF16 averages.

Why post-training quantization hurts

Standard 4-bit compression, known as post-training quantization or PTQ, takes a finished model and rounds its weights down to lower precision after the fact. The model never got a chance to adapt to the rounding error during training, so quality drops, and the drop is worst on small models where every parameter is already doing heavy lifting.

QAD flips that around. It simulates quantization during training and teaches the resulting quantized student to match a full-precision teacher. The student learns to be good while being small, rather than being made small after the fact. Crucially, the final artifact is still a standard Q4_0 GGUF, so no custom kernels or specialized runtime are required.

How much quality actually comes back

Liquid measured recovery as the fraction of the BF16-to-Q4_0 quality gap that QAD closes across a benchmark suite covering GPQA Diamond, MMLU-Pro, IFEval, IFBench, Multi-IF, and BFCLv4, plus a math eval (GSM8K for the small models, AIME25 for the larger ones). The numbers:

• QAD closes 70.6% of the gap for LFM2.5-230M, 73.4% for LFM2.5-350M, 65.5% for LFM2.5-1.2B, and 48.4% for LFM2.5-2.6B.
• The resulting checkpoints retain 97.1%, 96.5%, 97.4%, and 96.6% of their respective BF16 baseline performance.
• The strongest recovery lands on the smallest models, where PTQ hurts the most.

Speed on real hardware

The team benchmarked llama.cpp decode throughput across four backends: an Apple M5 Max MacBook Pro, an AMD Ryzen AI Max+ 395 NucBox EVO-X2, a Snapdragon 8 Elite Gen 5 Galaxy S26 Ultra, and a Raspberry Pi 5. Because QAD Q4_0 uses the same tensor layout and runtime path as native Q4_0, it inherits Q4_0 speed while pulling quality up toward higher-precision formats.

The comparisons against other 4-bit and 5-bit formats are where this gets practically interesting:

• The 230M and 350M QAD Q4_0 checkpoints match Q5_K_M quality within evaluation variance at 4-33% higher decode throughput.
• The 1.2B and 2.6B QAD Q4_0 checkpoints match Q4_K_M quality at 3-14% higher throughput.
• QAD Q4_0 also matches Unsloth's UD-Q4_K_XL, a strong external post-training quantization checkpoint, while staying smaller.

Using it

All four QAD GGUFs are on Hugging Face under the standard LFM2.5 repos, alongside the older PTQ files. The Quantization-Aware Distillation (QAD) checkpoint is available as LFM2.5-2.6B-QAD-Q4_0.gguf. This is distinct from the post-training-quantized LFM2.5-2.6B-Q4_0.gguf; both use the GGUF Q4_0 format. Pull them with llama.cpp and go:

llama-cli -hf LiquidAI/LFM2.5-2.6B-GGUF \
  -c 4096 --color -i \
  --temp 0.1 --top-k 50 --repeat-penalty 1.1

Where this fits

Liquid has been aiming LFM2.5 squarely at the latency, privacy, and hardware constraints of the physical world: phones, laptops, cars, single-board computers. QAD moves the practical Pareto frontier for that setting. If you were previously reaching for Q5_K_M to keep quality acceptable, or shipping a bigger PTQ Q4 model to buy back accuracy, the QAD checkpoint gives you a smaller, faster file with matching scores. And because the format is unchanged Q4_0, integrations with existing GGUF pipelines need zero code changes.

The broader signal here is that quantization-aware training, long treated as an expensive research technique, is becoming a routine step in the release pipeline for edge-targeted models. When 97% of BF16 quality fits in a 4-bit file that runs on a Raspberry Pi, the calculus for what belongs in the cloud versus on the device shifts a little further toward the device.