Prime Intellect's IPO Stops AI Training Crashes That Derailed Long Runs

Prime Intellect has released prime-rl v0.10.0 and verifiers v0.4.0. The central change is INTELLECT Policy Optimization, or IPO, a new default reinforcement-learning loss designed to stabilize asynchronous training when the trainer and inference engine assign different probabilities to the same tokens. The release also expands native model support, consolidates evaluation tooling, and improves reported mixture-of-experts training throughput.

Trainer drift can derail long runs

Asynchronous reinforcement learning generates rollouts, the model’s sampled responses, in an inference engine such as vLLM while a separate trainer updates the model. A rollout may reach the trainer after several updates, so it reflects an older policy. Different GPU kernels in the inference and training stacks can introduce additional numerical differences.

The resulting trainer-inference drift changes the probability assigned to each sampled token. Large discrepancies make the training signal unreliable and can destabilize long-running, off-policy workloads.

IcePop, a loss previously used to control this drift, masks tokens when the ratio between trainer and inference probabilities leaves a configured range. Ratios can overstate small changes among rare tokens and understate large absolute changes among likely tokens. A move from 0.01 to 0.06 produces a 6x ratio despite changing by five percentage points, while a move from 0.90 to 0.20 produces a 0.22x ratio after a 70-point change.

IPO measures absolute probability drift

IPO masks a token when the absolute difference between its trainer and inference probabilities exceeds epsilon:

|ptrainer - pinference| > ε

The mask applies to probability changes in either direction and does not depend on the sign of the token’s advantage, which represents whether the sampled action performed better or worse than expected. Epsilon is the only masking parameter. An epsilon of 0.3, for example, allows a difference of up to 30 percentage points.

The method adapts the trust-region idea used by DPPO, where updates are constrained to keep a policy from moving too far. IPO applies that constraint to disagreement between the rollout and training systems.

Prime Intellect tested IPO on agentic software-engineering reinforcement learning with GLM-4.5-Air, without replaying mixture-of-experts router decisions and with trajectories up to 64 steps off-policy. IPO completed the run with epsilon values of 0.2 and 0.3.

At the new default of eps = 0.3, IPO masked about 0.25% of tokens in the reported test. An epsilon of 0.1 masked about 2%. IcePop, PPO, and CISPO remain available as explicit loss selections.

MoE training reaches 3,732 tokens per second

Prime Intellect reports a 1.15× throughput lead (approximately 13.1% faster) over torchtitan in a back-to-back supervised fine-tuning benchmark for Qwen3-235B-A22B. The test used four nodes with eight NVIDIA B300 GPUs each, a 16,000-token sequence length, Fully Sharded Data Parallel training, and eight-way expert parallelism.

Model FLOPs utilization, or MFU, estimates how much of a GPU’s theoretical arithmetic capacity the workload uses. Mixture-of-experts models complicate that calculation and the underlying execution because each token activates only a subset of the model’s expert layers.

The release attributes the throughput improvement to several MoE-specific changes:

• A unified MoE runtime and checkpoint layout now serves every supported model.
• Gate, up, query, key, and value projections are fused at runtime.
• MoE blocks compile as complete graphs, reducing graph breaks and launch overhead.
• Loss computation uses a chunked implementation based on torchtitan’s approach.
• A SonicMoE BF16 backend adds per-layer selection of the MoE compute implementation.

GLM-5.3 receives a separate sparse-attention path. FlashMLA handles the forward pass, while cuDNN handles the backward pass. Prime Intellect reports that the forward implementation runs 1.9 to 3.6 times faster than TileLang on H200 GPUs and that the backward implementation runs 3 to 4.7 times faster.

A 12-node GLM-5.3 recipe with a 16,000-token sequence length removes context parallelism and offloading. In the reported full-model SFT benchmark, throughput increased from 465 to 835 tokens per second per GPU, while MFU rose from 22.3% to 40.1%.

Native coverage expands

Every architecture supported by prime-rl now uses a native implementation. Version 0.10 adds the following model families and capabilities:

• DeepSeek V4 Flash: FlashMLA and cuDNN sparse-attention kernels, plus ring context parallelism.
• Qwen3.8 Flash Next: n-gram embeddings sharded by vocabulary row across data-parallel ranks, indexed sparse attention, Gated DeltaNet, hyper-connection residuals, and sigmoid-gated MoE layers.
• Nemotron-H Nano and Super: Mamba-2 state resets at packed-document boundaries and Ulysses context parallelism.
• Multimodal RL: built-in support for Qwen-VL and Kimi K2.5.

Multimodal rollouts keep image references inline through generation and sequence packing. Earlier pipelines commonly transferred decoded pixel tensors between stages, creating a separate data path for images.

Evaluation moves into prime-rl

Evaluation now runs from prime-rl through uv run eval. The command supports standalone evaluation and online evaluation during training, with resumable traces and task selection shared with vf-eval.

Verifiers 0.4.0 completes the project’s transition to its v1 API stack. The release removes the legacy v0 implementation and standardizes command names under the vf-* prefix, so scripts that call older commands require updates.

What changes for existing users

• Training configurations that rely on the default loss will use IPO with eps = 0.3.
• IcePop, PPO, and CISPO must be selected explicitly when those behaviors are required.
• Evaluation scripts should invoke the prime-rl evaluation command.
• Verifiers integrations using the v0 stack or older CLI names must migrate to the v1 APIs and vf-* commands.

Both releases are available under open-source licenses. Prime Intellect provides additional benchmark details, configuration notes, and migration information in the release post.