---
title: "EOS Automatically Finds Optimal Vision Token Cuts in Just 3 GPU-Hours"
subtitle: "A multi-institution research team unifies visual token pruning, merging, pooling, and reweighting into one differentiable search space — and lets gradient descent pick the best operator automatically."
url: "https://alphasignal.ai/news/eos-automatically-finds-optimal-vision-token-cuts-in-just-3-gpu-hours"
author: "AlphaSignal Newsroom"
author_url: "https://alphasignal.ai/editorial-team"
company: "AlphaSignal"
company_url: "https://alphasignal.ai/company/alphasignal"
published: "2026-06-03T00:00:00.000Z"
updated: "2026-06-11T03:16:49.956Z"
type: "Paper"
topics:
  - "Llms"
  - "Infra"
subtopics:
  - "Vision Language"
  - "Structured Output"
  - "Inference Optimization"
upvotes: 2183
source_url: "https://arxiv.org/abs/2606.05232"
comments: 0
truncated: true
---

# EOS Automatically Finds Optimal Vision Token Cuts in Just 3 GPU-Hours

> A multi-institution research team unifies visual token pruning, merging, pooling, and reweighting into one differentiable search space — and lets gradient descent pick the best operator automatically.

Every team building a multimodal model eventually faces the same bottleneck: vision encoders produce hundreds or thousands of tokens per image, and feeding all of them into a large language backbone is expensive. The standard playbook is to pick a token-reduction operator , prune the unimportant ones, merge similar ones, pool spatially, or softly reweight , and hand-tune it. A new paper from researchers at the University of Sydney, ByteDance, Shanghai Jiao Tong University, and UCLA argues that this manual selection process is unnecessary, and introduces a framework called [Efficient Operator Search (EOS)](https://arxiv.org/abs/2606.05232) that finds the right operator automatically using gradients.

## The problem with picking operators by hand

Scaling multimodal foundation models has led to substantial performance gains, but these gains often come with substantial inference cost, as dense visual tokens are repeatedly processed by large language backbones, especially in high-resolution, multi-image, and long-context scenarios. The community has responded with a wave of hand-crafted solutions. Existing work improves the efficiency of foundation models through human-designed token reduction operators, including pruning, merging, pooling, and adaptive reweighting.

The trouble is that each of these operators embodies a different assumption about which tokens matter and how to handle the ones you drop. Pruning discards tokens entirely. Merging fuses similar tokens into a weighted average. Pooling applies a fixed spatial aggregation. Reweighting redistributes attention mass without removing tokens. Researchers pick one, tune it for a specific model and task, and call it done , but there is no principled reason to believe any single operator is universally best. Efficient multimodal foundation models often rely on manually designed token-reduction operators, such as pruning, merging, pooling, and adaptive reweighting. Although these operators appear different, they can be interpreted as distinct regimes of a shared operator space.

_Preview only. Full text requires an AlphaSignal Pro account._

## Sources

- [Differentiable Efficient Operator Search (arXiv)](https://arxiv.org/abs/2606.05232)
- [Differentiable Efficient Operator Search (HTML full paper)](https://arxiv.org/html/2606.05232)
- [Awesome-Collection-Token-Reduction (GitHub)](https://github.com/ZLKong/Awesome-Collection-Token-Reduction)
- [Rethinking Token Reduction in MLLMs: Towards a Unified Paradigm](https://arxiv.org/html/2411.17686v1)
- [Token Reduction Should Go Beyond Efficiency in Generative Models](https://arxiv.org/html/2505.18227v1)