Liquid AI's LFM2.5 Hits 600 Tok/s but Fails Every Quality Check

A community checkpoint on Hugging Face converts Liquid AI’s mixture-of-experts reasoning model into a block masked-diffusion sampler. Published by summerAIO, diffusion-lfm2.5-24b-a2b is an unofficial LoRA-merged fine-tune that reports more than 600 output tokens per second on one A100. Its generations fail the author’s basic quality checks, making the release a documented experiment rather than a production model.

An MoE rewired for diffusion

The upstream checkpoint contains 24 billion parameters, with 2.45 billion active for each token. Its mixture-of-experts blocks contain 64 experts and route each token through the top four. The 40-layer architecture combines 30 double-gated short-convolution blocks with 10 grouped-query attention layers and supports a 131K-token context window.

SummerAIO trained a masked-diffusion LoRA adapter with rank 64 and alpha 128, then merged it into the BF16 base weights and packed the expert weights. LoRA updates a compact set of low-rank parameters, reducing fine-tuning costs compared with updating every model weight. Training used Alpaca and ARC-Easy data for 3,100 update steps, with the final checkpoint selected by validation masked-token negative log likelihood.

The merged artifact occupies about 48GB on disk. It relies on a custom modeling_diffusion_lfm.py implementation, so standard Transformers generation code will not run unchanged. Integrators need the repository’s sampler and should audit the custom modeling code before allowing a loader to execute it.