Skip to main content
Back

OVHcloud's Olala Combines Mamba-3 and Attention to Run Locally on 6GB

OVHcloud's AI arm released Olala mini, an Apache 2.0 hybrid Mamba-3 and attention MoE that fits on a laptop and reasons before answering.

Model, Full Access
  • OVHai LLM (OVHcloud) released Olala-v1.0-mini-7A1B, a 6.8B total / ~1B active MoE reasoning model under Apache 2.0.
  • Hybrid architecture: 29 Mamba-3 layers plus 7 Grouped Differential Attention layers, each with a 256-expert LatentMoE block.
  • Trained on 12.8T tokens across JUPITER, Jean Zay and MeluXina supercomputers; averages 68.0 across 12 benchmarks.
  • Four reasoning-effort levels, 32K context, native tool calling, 37 languages, trained for agentic multi-turn tool use up to 75 turns.
  • Only 7 of 36 layers keep a KV cache, with a 6x smaller cache via Tensor Product Attention, enabling laptop inference.
  • Served via OVHcloud AI Endpoints, Hugging Face Inference Providers and Blade; vLLM, llama.cpp and MLX need OVHai forks.

OVHcloud’s in-house model team, OVHai LLM, has released Olala-v1.0-mini-7A1B, an open-weights reasoning model designed for local chat, tool calling, and multi-turn agent workflows. Its 6.8 billion parameters combine Mamba-3 state-space layers, a custom attention design, and mixture-of-experts feed-forward blocks.

The Apache 2.0 release was trained on European supercomputers and supports a 32K-token context window. Across 12 benchmarks, the model card reports an average score of 68.0, up 15.2 points from the supervised fine-tuning checkpoint. Its architecture offers the more useful result: low active compute and a smaller context-dependent cache in a model that can run on quantized laptop builds.

A 7B shell with a 1B route

Specification Olala-v1.0-mini-7A1B
Total parameters 6.8B
Active parameters per token About 1B
Backbone 29 Mamba-3 layers and 7 attention layers
Experts per layer 256 routed experts, with 6 selected per token, plus 1 shared expert
Context window 32K tokens; the team says a 128K version is planned
Language coverage English and 36 other languages
License Apache 2.0 weights

All 6.8 billion weights must fit in memory, while expert routing limits each token’s compute path to about 1 billion parameters. That distinction explains why quantization remains important for laptop deployment even though inference compute resembles a much smaller dense model.

Seven attention layers carry the growing cache

A transformer’s key-value cache stores representations for previous tokens at every attention layer, so its memory use grows with context length and batch size. Olala restricts that growing cache to seven of its 36 layers. The remaining 29 Mamba-3 layers compress sequence history into fixed-size recurrent states during autoregressive decoding.

Diagram of Olala’s 29 Mamba-3 layers, seven attention layers, and mixture-of-experts blocks
Olala alternates mostly Mamba-3 layers with seven attention layers, each followed by a mixture-of-experts block.

Lead researcher Alexandre explains in a research thread that the Mamba layers handle general sequence mixing, while attention specializes in retrieving precise earlier information. The team combines four techniques in its attention block:

  • Grouped Differential Attention: Three-quarters of the heads model useful signal and one-quarter estimates noise, which is subtracted from the attention logits.
  • Scalable Softmax: Rescales logits to limit attention diffusion as the context grows.