---
title: "Frozen Qwen2.5-VL Beats Trained Image Search Models Without a Single Gradient Step"
subtitle: "A new paper turns off-the-shelf multimodal LLMs into zero-shot image-to-image re-rankers, beating task-specific models on out-of-domain retrieval."
url: "https://alphasignal.ai/news/frozen-qwen2-5-vl-beats-trained-image-search-models-without-a-single-gradient"
author: "AlphaSignal Newsroom"
author_url: "https://alphasignal.ai/editorial-team"
company: "AlphaSignal"
company_url: "https://alphasignal.ai/company/alphasignal"
published: "2026-04-14T00:00:00.000Z"
updated: "2026-08-23T22:35:14.729Z"
type: "Paper"
topics:
  - "Image"
  - "Retrieval"
subtopics:
  - "Vision Language"
  - "Image Generation"
  - "Reranking"
upvotes: 1779
source_url: "https://arxiv.org/abs/2604.13268"
comments: 0
truncated: true
---

# Frozen Qwen2.5-VL Beats Trained Image Search Models Without a Single Gradient Step

> A new paper turns off-the-shelf multimodal LLMs into zero-shot image-to-image re-rankers, beating task-specific models on out-of-domain retrieval.

Multimodal LLMs are usually pitched as chat interfaces for images. A new paper from Mohamed bin Zayed University of AI and Czech Technical University in Prague flips that framing: it uses a frozen [Qwen2.5-VL-7B](https://arxiv.org/abs/2604.13268) as a training-free similarity engine for instance-level image retrieval, and shows it can outperform re-rankers built specifically for the job, without a single gradient step.

The mechanism is disarmingly simple. Prompt the model with paired images, ask whether they show the same object instance, and require a single-digit reply. The softmax probability assigned to the token "1" becomes the similarity score. That is the whole re-ranker.

## Why bother with an LLM for pixels?

Instance-level retrieval is the fine-grained variant of image search: not "find another dog," but "find this exact dog." State-of-the-art systems typically bolt a re-ranking stage onto a global-descriptor search, using methods like RRT, CVNet, or AMES that cross-compare local patch features. They work, but they need domain-specific training data and tend to collapse outside their training distribution. AMES, for instance, is trained on the Google Landmarks dataset and excels at buildings while faltering elsewhere.

Foundation models like CLIP and DINOv3 give strong global descriptors, though global vectors lose precision on small objects, occlusion, and clutter. The authors argue that MLLMs already learned rich visual discrimination during pre-training on massive multimodal corpora, so you should be able to elicit that as a similarity function through prompting alone.

_Preview only. Full text requires an AlphaSignal Pro account._

## Sources

- [Indexing Multimodal Language Models for Large-scale Image Retrieval](https://arxiv.org/abs/2604.13268)
- [arXiv HTML version of the paper](https://arxiv.org/html/2604.13268)
- [MLLM-ILR code repository](https://github.com/gkordo/mllm-ilr)
- [ILIAS: Instance-Level Image retrieval At Scale](https://vrg.fel.cvut.cz/ilias/)