---
title: "Huawei's rDPO Beats Standard Vision AI Training With Per-Image Scoring Checklists"
subtitle: "Huawei's rDPO framework builds per-image checklists that turn a mid-sized open judge into a GPT-5.4 rival and lifts vision model alignment across the board."
url: "https://alphasignal.ai/news/huawei-s-rdpo-beats-standard-vision-ai-training-with-per-image-scoring"
author: "AlphaSignal Newsroom"
author_url: "https://alphasignal.ai/editorial-team"
company: "AlphaSignal"
company_url: "https://alphasignal.ai/company/alphasignal"
published: "2026-04-14T00:00:00.000Z"
updated: "2026-08-23T23:25:37.612Z"
type: "Paper"
topics:
  - "Post Training"
  - "Image"
subtopics:
  - "Dpo"
  - "Vision Language"
upvotes: 1033
source_url: "https://arxiv.org/abs/2604.13029"
comments: 0
truncated: true
---

# Huawei's rDPO Beats Standard Vision AI Training With Per-Image Scoring Checklists

> Huawei's rDPO framework builds per-image checklists that turn a mid-sized open judge into a GPT-5.4 rival and lifts vision model alignment across the board.

Direct Preference Optimization has become the default recipe for teaching vision-language models to answer more accurately and hallucinate less, but the recipe is only as good as the preference pairs you feed it. A team at Huawei argues that the pairs most pipelines use today are too blunt for visual reasoning, and they have released a framework called [rDPO](https://arxiv.org/abs/2604.13029) that swaps scalar judgments for per-image checklists.

The core idea is straightforward. For every image-instruction pair in the training pool, an offline pipeline drafts a rubric of concrete, weighted criteria that any candidate answer must satisfy. A judge model then scores rollouts from the target policy against that rubric, and only pairs with a clear, criterion-level quality gap become DPO training data. On a comprehensive benchmark, rDPO reaches 61.01, well ahead of a style-constrained baseline at 52.36 and the 59.48 base model, suggesting that visual preference optimization benefits from combining on-policy data construction with instance-specific criterion-level feedback.

## Why coarse rewards break down on images

Most VLM preference pipelines fall into one of two camps. Response-oriented methods inject hallucinations or contrast strong and weak model outputs. Vision-oriented methods perturb the image itself with diffusion noise or edits. Both tend to produce off-policy pairs that drift away from what the target model would actually generate, which weakens the DPO gradient signal.

Newer on-policy pipelines like RLAIF-V and MMPR sample from the model being trained, but they still rely on indirect signals that fall short of the fine-grained feedback needed for complex visual reasoning. Grouping responses by hallucination count or final correctness misses the more interesting failure modes: partial grounding, missing reasoning steps, or an answer that is technically right but skips the visual evidence.

Rubric-based evaluation has already gained traction in the text-only world through work like Rubrics-as-Rewards and OpenRubrics, but on the multimodal side it has remained largely template-driven. rDPO's contribution is generating criteria specific to every single instance rather than composed from a fixed pool.

_Preview only. Full text requires an AlphaSignal Pro account._

## Sources

- [Visual Preference Optimization with Rubric Rewards](https://arxiv.org/abs/2604.13029)
- [Visual Preference Optimization with Rubric Rewards (HTML)](https://arxiv.org/html/2604.13029)
- [Rubric-Driven DPO for Visual Tasks](https://www.startuphub.ai/ai-news/ai-research/2026/rubric-driven-dpo-for-visual-tasks)
- [Policy-Aware Rubric Reward (POW3R)](https://www.emergentmind.com/topics/policy-aware-rubric-reward-pow3r)