LocalAI Squeezes LFM2.5 to 3.6 GB Without Losing Its Reasoning Power
The LocalAI team applies APEX, a MoE-aware quantization method, to Liquid AI's LFM2.5-8B-A1B, squeezing a 128K-context reasoning model down to 3.6–6.3 GB for fully local inference.
Takeaways
- LFM2.5-8B-A1B APEX GGUF is now available on Hugging Face, compressing Liquid AI's hybrid MoE model to 3.6–6.3 GB for local inference.
- APEX quantization assigns different precision per tensor role and layer position, beating Q8_0 quality at roughly half the file size.
- LFM2.5-8B-A1B activates only 1.5B of 8.3B parameters per token, supports 128K context, and is a reasoning-only model with explicit chain-of-thought.
- Benchmark gains over its predecessor are large: IFEval +12 pts, AIME25 +22 pts, non-hallucination rate jumps from 7% to 63%.
- Seven file tiers from 3.6 GB (Mini) to 6.3 GB (Balanced) cover everything from phones to consumer GPUs; all work with llama.cpp, Ollama, and LocalAI out of the box.
- Free to use — the GGUF files are open-weight; the APEX quantization tool is MIT-licensed and works on any GGUF-compatible MoE model.
Liquid AI's LFM2.5-8B-A1B is already one of the most interesting small models around: a non-transformer hybrid that fits on a laptop and beats models several times its size on agentic benchmarks. Now the LocalAI team has taken it a step further, releasing LFM2.5-8B-A1B-APEX-GGUF , a set of quantized GGUF files that compress the model down to as little as 3.6 GB while preserving nearly all of its quality, using their own APEX quantization technique built specifically for Mixture-of-Experts architectures.
What makes LFM2.5 worth quantizing
LFM2.5 is a new family of hybrid models designed for on-device deployment, building on the LFM2 architecture with extended pre-training and reinforcement learning. The architecture is genuinely unusual: it has 24 layers , 18 double-gated LIV convolution blocks and 6 GQA layers. Those short-convolution blocks replace most of the attention layers you'd find in a standard transformer, which is the key reason the model sidesteps the KV cache memory explosion that plagues long-context inference.
The architecture totals 8.3B parameters but activates only 1.5B per token, enabling reasonable throughput and memory use on CPUs, laptops, and phones. That sparse activation is the MoE (Mixture-of-Experts) part , the model has 32 routed experts per layer but only fires 4 of them for each token it processes. LFM2.5-8B-A1B is the fastest model in its size class, reaching 18.5K output tokens per second at high concurrency, over 1.6B tokens per day on a single H100.
The .5 in the name isn't cosmetic. Compared to LFM2-8B-A1B, this version expands the context window from 32,768 to 128,000 tokens, allowing the model to process longer documents and reason for longer. Unlike its predecessor, LFM2.5-8B-A1B is a reasoning-only model, producing an explicit chain of thought before its final answer. Liquid AI adopted this strategy because MoE models generally run in compute-bound settings, where a smaller number of active parameters makes each reasoning token cheap , providing a significant quality boost without compromising speed.
The numbers behind the upgrade
The benchmark jump from LFM2 to LFM2.5 is steep. On IFEval (instruction following), the score goes from 79.44 to 91.84. On AIME25 (competition math), it goes from 20.00 to 42.53. The most dramatic improvement is on hallucination: the non-hallucination rate jumps from 7.46% to 63.47%, a result of a dedicated RL stage that reinforces abstention when the model doesn't know something rather than letting it confabulate.