UIUC and Alibaba's PAPO Fixes the Blind Spot Killing Vision AI Reasoning
PAPO adds a perception-aware KL term to GRPO and DAPO, cutting multimodal reasoning errors by up to 30% without new data or reward models.
Takeaways
- PAPO adds a perception-aware KL loss to GRPO/DAPO, boosting multimodal reasoning by 4.4-17.5% overall.
- Manual error analysis found 67% of GRPO errors on VLMs come from perception, not reasoning.
- The Implicit Perception Loss maximizes KL divergence between outputs given original vs randomly masked images.
- Double Entropy Loss regularization prevents a novel KL-hacking collapse mode in larger models.
- Vision-dependent task gains reach 8-19.1%, with a 30.5% reduction in perception errors.
- Code, data, and trained checkpoints are open-sourced at GitHub and Hugging Face.
Reinforcement learning with verifiable rewards has been the engine behind reasoning breakthroughs in text-only models like DeepSeek-R1. Plug the same recipe into vision language models and the gains get muted. A team from UIUC and Alibaba dug into why, and their answer is uncomfortable: the models are not failing at reasoning. They are failing at seeing.
Their new algorithm, PAPO (Perception-Aware Policy Optimization), is a drop-in replacement for GRPO and DAPO that forces the policy to actually rely on the image while it reasons. The paper reports overall gains of 4.4% to 17.5% across eight multimodal benchmarks, with a 30.5% reduction in perception errors, and no new data, reward models, or teacher models required.
The 67% problem nobody was solving
The authors trained Qwen2.5-VL-3B with a standard GRPO pipeline on ViRL39K, then hand-labeled 200 error cases across four benchmarks. They found that 67% of errors stemmed from perception, not reasoning, calculation, or inconsistency. In many failures the model executed the algebra correctly but misread a label, misassigned a variable to the wrong side of a triangle, or hallucinated a spatial relationship.

That is a pointed diagnosis. Most prior work on multimodal RLVR has focused on three levers: better data curation, more diverse rollouts, or richer reward design (for example, adding captioning rewards or IoU rewards for grounding). The core optimization objective inherited from text-only RL was left untouched. PAPO argues that is the actual bug, because nothing in the GRPO objective explicitly rewards the model for grounding its output in the pixels.