Prime Intellect Unifies 365,000 Agentic Tasks Into one Open RL Training Framework
Prime Intellect unifies 365,000+ validated agentic RL tasks across SWE, terminal, and search behind a single API and image registry
- 365,000+ tasks unified: Prime Intellect ships 23 agentic tasksets (SWE, terminal, search) behind one API, one image registry, and one command.
- 135,000 prebuilt Docker images hosted in Prime's own registry -- the largest such catalog by any sandbox provider, eliminating Docker Hub rate limits at scale.
- Rigorous validation pipeline: Every default dataset passed gold-patch and no-op validation with up to 10x retries; cleaned versions are re-uploaded to Hugging Face with full audit trails.
- Reward hacking mitigated by design: Grading scripts, test patches, and expected outputs are withheld from the agent sandbox until scoring time.
- Plugs into prime-rl: GLM-4.5-Air trained on ScaleSWE on 6 H200 nodes in 2 days, with eval Pass@1 on SWE-Bench-Verified rising from 0.40 to 0.56.
- Open and extensible: All environments live in the research-environments repo; adding a new taskset requires only a small contract class.
Training an RL agent to write code, navigate terminals, or search the web has always meant wrestling with a different harness, a different Docker setup, and a different grading script for every dataset you want to use. Prime Intellect's new release cuts through that fragmentation: 23 agentic tasksets, 365,000+ tasks, one API, one image registry, one command.
Why fragmentation persisted
Popular evaluation suites like lm_eval and HELM handle single-turn Q&A evals well, but lack support for agentic tasks that require complex infrastructure like TAU-bench, TerminalBench, and SWE-bench. The result has been a sprawl of independent eval repos with no shared entrypoints or specs. Every team building a coding or search agent rebuilt the same scaffolding from scratch.
RL environments and agent evals are fundamentally the same thing: a dataset, a harness, and scoring rules. Open-source efforts have treated them as separate problems. Prime Intellect's answer is a unified abstraction called verifiers v1, which decomposes any environment into three independent layers: a taskset, a harness, and a runtime. This release delivers the taskset layer.
What's in the catalog
Twenty-three agentic tasksets ship behind one taskset API: roughly 198,000 software engineering tasks across 20+ languages, 28,600 terminal tasks, and 137,600 search tasks, totaling around 365,000 tasks ready for evals and RL training.
The three domains each have a distinct character:
- Software engineering (SWE): Covers the full spectrum from canonical benchmarks to large training corpora. Includes SWE-bench Verified (500 human-filtered GitHub issues), SWE-smith (88,130 synthetically injected bugs across 8 languages), SWE-rebench-V2 (32,079 freshly mined PRs across 20 languages), OpenSWE (45,320 tasks), and more.