DeepSeek AI's DeepSeekMath-7B Beats 540B Models and Birthed GRPO
DeepSeekMath-7B-Instruct brings near-GPT-4 math reasoning to a 7B open-source model using a novel RL algorithm that's now reshaping how reasoning models are trained.
PRO- Near-GPT-4 math at 7B: DeepSeekMath-7B-Instruct scores 51.7% on the competition-level MATH benchmark, surpassing all open-source models and most proprietary ones.
- GRPO introduced here: The paper introduced GRPO, the RL algorithm that later powered DeepSeek-R1 and Qwen2.5 reasoning models.
- 120B token math corpus: A custom iterative FastText pipeline mined 35.5M math web pages from Common Crawl, 7x larger than Minerva's dataset.
- No critic model needed: GRPO replaces PPO's expensive value network with group-relative reward normalization, cutting memory and compute costs.
- Free and commercial-use: Available on Hugging Face with MIT-licensed code and a permissive model license.
- Weaknesses remain: Geometry, formal theorem proving, and few-shot tasks still lag behind GPT-4 and closed models.
DeepSeekMath-7B-Instruct is an open-source, instruction-tuned language model from DeepSeek AI built specifically for mathematical reasoning. At just 7 billion parameters, it punches far above its weight class, hitting scores that rival models many times its size. But the real story here is not just the benchmark numbers. It is the two innovations behind them, one in data engineering and one in reinforcement learning, that have since become foundational techniques across the entire field of reasoning-focused LLMs.
A 7B model that nearly matches GPT-4 on competition math
DeepSeekMath-7B continues pre-training DeepSeek-Coder-Base-v1.5 7B with 120B math-related tokens sourced from Common Crawl, together with natural language and code data, achieving 51.7% on the competition-level MATH benchmark without relying on external toolkits or voting techniques. That is a remarkable number for a 7B model. DeepSeekMath-Base 7B outperforms Minerva 540B on several benchmarks, demonstrating that careful data curation and training methodology can be more effective than simply scaling model size.
On the competition-level MATH dataset, the instruct model surpasses all open-source models and the majority of proprietary models by at least 9% absolute, including models that are substantially larger such as Qwen 72B, or those specifically enhanced through math-focused reinforcement learning like WizardMath-v1.1 7B. Push it further with sampling and self-consistency over 64 samples achieves 60.9% on MATH.
Innovation #1: Mining the web for math at scale
The first key ingredient is a purpose-built data pipeline. The DeepSeekMath Corpus is a high-quality dataset of 120B tokens from web pages filtered for mathematical content, almost 7 times the size of the math web pages used by Minerva and 9 times the size of the recently released OpenWebMath.
The pipeline works iteratively. It starts by using OpenWebMath as seed data to train a FastText classifier, then uses that classifier to retrieve math pages from Common Crawl, identifies math-related domains through statistical analysis, manually annotates URLs within those domains, and adds newly discovered pages back to the seed. After four iterations, the result is 35.5 million mathematical web pages totaling 120B tokens.
This story is for Pro members
You've reached the end of the free preview. Upgrade to AlphaSignal Pro to read the full article - and everything else behind the paywall.
Comments
