Skip to main content
Back

Xiaohongshu Turns Qwen2.5-VL-7B Into a Multi-Image Grounder Beating 72B Models

A two-stage recipe combining chain-of-thought fine-tuning with GRPO reinforcement learning pushes a 7B model past 72B baselines on cross-image grounding tasks.

Pro

Paper, Full Access
  • New RL pipeline improves multi-image grounding for MLLMs, +9.04% on MIG-Bench over SFT baseline.
  • Two stages: LoRA CoT-SFT cold start using distilled Qwen2.5-VL-72B rationales, then GRPO reinforcement learning.
  • Rejection sampling between stages removes zero-variance batches, adding 3.27% and stabilizing RL training.
  • 7B model beats Qwen2.5-VL-72B by 12.60% on MIG-Bench referential grounding subsets.
  • Generalizes across 7 out-of-domain benchmarks including ODINW, LISA, BLINK, MMIU with +4.41% average gain.
  • Paper builds on Migician's MGrounding-630k dataset; no public checkpoint released yet.

Visual grounding, the task of drawing a bounding box around what a text description refers to, has been largely solved for single images. Real-world queries rarely stay that neat. You often want to point at something in one image based on context from another, and that is where today's multimodal models fall apart. A new paper from researchers at Xiaohongshu, USTC, and collaborators tackles this with a reinforcement learning pipeline that turns Qwen2.5-VL-7B into a state-of-the-art multi-image grounder, and the full method is worth a look.

The cross-image blind spot

Traditional visual grounding localizes targets in a single image from simple language descriptions, but real applications demand multi-image grounding with complex instructions and cross-image reasoning. Consider a query like "find the object in Image-2 that the person in Image-1 most needs," which requires the model to infer intent from one picture and then localize the answer in another.

The prior state of the art, Migician, introduced a multi-image grounding dataset and used a two-stage supervised fine-tuning pipeline. Borrowing framing from a recent Berkeley paper the authors cite, SFT tends to memorize understanding patterns and instruction-following behaviors rather than cultivating reasoning, which limits generalization. Naively throwing GRPO reinforcement learning at Qwen2.5-VL-7B did not work either, because the base model frequently failed to generate correct responses during RL training due to limited capacity for complex cross-image queries. No correct rollouts means no reward signal to learn from.