OVHcloud's Olala Combines Mamba-3 and Attention to Run Locally on 6GB
OVHcloud's AI arm released Olala mini, an Apache 2.0 hybrid Mamba-3 and attention MoE that fits on a laptop and reasons before answering.
Takeaways
- OVHai LLM (OVHcloud) released Olala-v1.0-mini-7A1B, a 6.8B total / ~1B active MoE reasoning model under Apache 2.0.
- Hybrid architecture: 29 Mamba-3 layers plus 7 Grouped Differential Attention layers, each with a 256-expert LatentMoE block.
- Trained on 12.8T tokens across JUPITER, Jean Zay and MeluXina supercomputers; averages 68.0 across 12 benchmarks.
- Four reasoning-effort levels, 32K context, native tool calling, 37 languages, trained for agentic multi-turn tool use up to 75 turns.
- Only 7 of 36 layers keep a KV cache, with a 6x smaller cache via Tensor Product Attention, enabling laptop inference.
- Served via OVHcloud AI Endpoints, Hugging Face Inference Providers and Blade; vLLM, llama.cpp and MLX need OVHai forks.
OVHcloud’s in-house model team, OVHai LLM, has released Olala-v1.0-mini-7A1B, an open-weights reasoning model designed for local chat, tool calling, and multi-turn agent workflows. Its 6.8 billion parameters combine Mamba-3 state-space layers, a custom attention design, and mixture-of-experts feed-forward blocks.
The Apache 2.0 release was trained on European supercomputers and supports a 32K-token context window. Across 12 benchmarks, the model card reports an average score of 68.0, up 15.2 points from the supervised fine-tuning checkpoint. Its architecture offers the more useful result: low active compute and a smaller context-dependent cache in a model that can run on quantized laptop builds.
A 7B shell with a 1B route
| Specification | Olala-v1.0-mini-7A1B |
|---|---|
| Total parameters | 6.8B |
| Active parameters per token | About 1B |
| Backbone | 29 Mamba-3 layers and 7 attention layers |
| Experts per layer | 256 routed experts, with 6 selected per token, plus 1 shared expert |
| Context window | 32K tokens; the team says a 128K version is planned |
| Language coverage | English and 36 other languages |
| License | Apache 2.0 weights |
All 6.8 billion weights must fit in memory, while expert routing limits each token’s compute path to about 1 billion parameters. That distinction explains why quantization remains important for laptop deployment even though inference compute resembles a much smaller dense model.
Seven attention layers carry the growing cache
A transformer’s key-value cache stores representations for previous tokens at every attention layer, so its memory use grows with context length and batch size. Olala restricts that growing cache to seven of its 36 layers. The remaining 29 Mamba-3 layers compress sequence history into fixed-size recurrent states during autoregressive decoding.
Lead researcher Alexandre explains in a research thread that the Mamba layers handle general sequence mixing, while attention specializes in retrieving precise earlier information. The team combines four techniques in its attention block:
- Grouped Differential Attention: Three-quarters of the heads model useful signal and one-quarter estimates noise, which is subtracted from the attention logits.
- Scalable Softmax: Rescales logits to limit attention diffusion as the context grows.