LiquidAI's LFM2.5-Encoder Beats ModernBERT at Long Context 3.7x Faster on CPU
Liquid AI's new bidirectional encoders run 3.7x faster than ModernBERT on CPU at 8k tokens, with top-4 accuracy across 14 competing models
- Liquid AI releases LFM2.5-Encoder-230M and LFM2.5-Encoder-350M, open-weight bidirectional encoders with 8,192-token context.
- LFM2.5-Encoder-230M is 3.7x faster than ModernBERT-base on CPU at 8k tokens (28s vs 90s per forward pass).
- LFM2.5-Encoder-350M ranks 4th of 14 models on GLUE/SuperGLUE/multilingual benchmarks, behind only three larger models.
- Built by converting LFM2 causal decoders into bidirectional encoders via attention mask flip, symmetric convolutions, and 30% masked-language training.
- Five live CPU-only demos available: prompt routing, policy linting, spell checking, PII detection, and a masked-diffusion chatbot.
- Available now on Hugging Face under LFM Open License v1.0; fine-tuning follows standard BERT-style recipes.
Encoders are the workhorses of production NLP. Every intent classifier, safety filter, and document router you run all day, every day, is almost certainly an encoder. LFM2.5-Encoder-230M and LFM2.5-Encoder-350M are Liquid AI's first general-purpose encoders, and they come with a very specific pitch: fast at long context, even without a GPU.
The problem with existing encoders at long context
Encoders power many modern production NLP applications: classifiers, intent routers, safety filters. These jobs run all day, usually on CPU, on ever-longer inputs. The trouble is that standard transformer-based encoders like BERT and ModernBERT have attention costs that scale quadratically with input length. Push them to 8,192 tokens on a CPU and they become unusably slow. At 8,192 tokens, ModernBERT-base takes over a minute and a half per forward pass versus about 28 seconds for LFM2.5-Encoder-230M. That's a 3.7x gap, and it's the core reason this release matters.
Built from a decoder, flipped for understanding
Liquid AI initializes the encoders from their respective LFM2 decoder backbones, then turns each causal decoder into a bidirectional encoder with a few targeted changes. The LFM2 backbone is a hybrid architecture that interleaves gated short-convolution blocks with grouped-query attention. Convolutions are cheap and scale linearly with sequence length, which is why the cost stays manageable at 8k tokens.