Renmin University's LMM-Searcher Lets AI Browse Images Without Exploding Memory
A file-based memory system lets multimodal agents scale to 100 search turns without drowning in image tokens, setting new open-source records.
Takeaways
- LMM-Searcher stores retrieved images as external files and references them via textual UIDs to prevent context explosion.
- A
fetch_imagetool loads pixels on demand, decoupling heavy visual perception from lightweight long-horizon reasoning. - Fine-tunes Qwen3-VL-30B-A3B-Thinking on 12K trajectories synthesized via a multi-hop knowledge graph pipeline.
- Achieves 30.1 on MM-BrowseComp and 34.8 on MMSearch-Plus at 100 turns, top open-source scores.
- Merged with MiroThinker-1.7-mini text agent (alpha=0.8) to transfer long-horizon search skills to multimodal setting.
- Full code released at github.com/RUCAIBox/LMM-Searcher, including framework, tools, and training setup.
Multimodal search agents keep hitting the same wall: every image they pull off a webpage explodes the context window. A team from Renmin University of China proposes a fix borrowed from how humans browse the web. Rather than stuffing every retrieved picture into the model's memory, their agent stores images as files and loads them only when it needs to look.
The system, called LMM-Searcher, is an open-source multimodal deep search agent built on Qwen3-VL-30B-A3B-Thinking. Across four benchmarks, it scales to 100-turn search horizons and achieves state-of-the-art performance among open-source models on challenging long-horizon evaluations like MM-BrowseComp and MMSearch-Plus.
Why multimodal agents choke
Language-only deep search agents like Search-R1 and R1-Searcher can chain dozens of queries before running out of room. Multimodal agents cannot. A single high-resolution webpage image can cost thousands of tokens, and a real research task might touch dozens of pages.
Prior systems tried two hacks, both bad. Some kept every image in context, causing the window to blow up within a handful of turns. Others aggressively discarded intermediate visuals, losing evidence the agent might need later. Text-style context compression does not port cleanly to visual data either, since pixels are not long strings you can summarize.
Treat images like files, not tokens
The core idea is to decouple perception from reasoning. LMM-Searcher offloads visual assets to an external file system, representing them within the context via lightweight textual identifiers (UIDs). In practice the UID is just the image URL. The agent's reasoning trace sees a compact reference like a filename plus a caption, while the actual pixels live on disk (or in cloud storage) until needed.