AllenAI Opens the Full Training Stack Behind Tulu 3 to Beat GPT-4o-mini

Most labs publish their base model weights and call the release open. AllenAI has been quietly doing the harder thing: releasing the entire post-training stack that turns a raw pretrained model into something you'd actually want to talk to. Their open-instruct repository is the working codebase behind the Tulu and OLMo instruction models, and it's one of the few places you can study a frontier-quality alignment pipeline end to end.

The full alignment stack in one repo

The repo serves as an open effort on instruction-tuning and post-training popular pretrained language models on publicly available datasets. Concretely, it bundles three things that usually live in separate research codebases:

• Code for finetuning language models with latest techniques and instruction datasets in a unified format
• Code for DPO, preference finetuning and reinforcement learning with verifiable rewards (RLVR)
• Checkpoints and other useful artifacts from AllenAI's exploration

RLVR is the piece worth pausing on. Instead of training a reward model to guess what humans like, you use programmatic checks (does the math answer equal the ground truth, does the code pass tests, does the output match a format) as the reward signal. The training methodology incorporates Supervised Fine-tuning, Direct Preference Optimization, and an innovative technique termed Reinforcement Learning with Verifiable Rewards, which ensures reliable and measurable improvements in model behavior.