MiniCPM5-2B targets local agents with a 128K context window

OpenBMB has released MiniCPM5-2B, a dense language model designed for local and edge inference, along with quantized builds, a speculative-decoding draft model, training data, and intermediate checkpoints. The model card reports an average benchmark score of 53.9, ahead of the strongest larger baseline in its comparison at 51.1. OpenBMB publishes the model and released training assets under the Apache 2.0 license.

The release combines a conventional Llama-compatible architecture with a native 131,072-token context window and post-training focused on reasoning, coding, tool use, and search. Several reported scores exceed those of Qwen3.5-4B and granite-4.2-3B, although the results come from OpenBMB’s evaluation and may vary with prompts, harnesses, inference settings, and agent scaffolding.

A familiar core with a long context

MiniCPM5-2B contains 2.5 billion total parameters, including 1.98 billion non-embedding parameters. As a dense model, it activates the full network for each token instead of routing tokens through a subset of experts. Its 42 transformer layers use grouped-query attention with 16 query heads and two key-value heads, which reduces key-value cache requirements relative to full multi-head attention.

The model follows the standard LlamaForCausalLM structure and requires no custom kernels or model-code fork. That compatibility allows established inference engines to load it through their existing Llama implementations, subject to using versions that recognize the model configuration.

Strong reported scores, uneven edges

OpenBMB’s evaluation highlights gains in code generation, mathematical reasoning, software engineering, tool use, long-context retrieval, and search-agent tasks.