Skip to main content
Back

STaR-KV Slashes UI-TARS GPU Memory by 40% Without Retraining

STaR-KV cuts peak GPU memory by 40% for GUI agents by rethinking how KV cache tokens are scored and pruned at inference time

  • Memory crisis: UI-TARS-1.5-7B uses 76 GB GPU memory on just 5 screenshots, nearly maxing out an 80 GB A100.
  • New method: STaR-KV is a training-free KV cache compression framework built specifically for GUI vision-language model agents.
  • Three innovations: Subspace-level spatial mutual information scoring, cumulative temporal stability discounting, and entropy-based adaptive score sharpening.
  • Key result: At a 20% KV-cache budget, STaR-KV cuts peak GPU memory by ~40% while outperforming GUIKV and SnapKV on four GUI benchmarks.
  • Zero overhead: Compression adds only -0.07% extra FLOPs and requires no fine-tuning -- plug directly into existing UI-TARS or OpenCUA checkpoints.
  • Open source: Code is available at github.com/kawhiiiileo/STaR-KV; evaluated on ScreenSpot-v2, ScreenSpot-Pro, AndroidControl, and AgentNetBench.

Running a GUI agent for more than a few steps is quietly becoming a hardware problem. A single task typically ingests tens of high-resolution screenshots as historical context, causing the KV cache to grow linearly with the number of interaction steps. The KV cache -- the memory structure that stores intermediate attention states so the model does not have to reprocess every past token -- is the culprit. Since GUI tasks typically involve high-resolution screenshots and extended interaction trajectories, the size of the KV cache grows linearly with sequence length, leading to severe GPU memory saturation and inference latency.

The numbers are stark. According to the paper, UI-TARS-1.5-7B -- ByteDance's state-of-the-art GUI agent -- consumes 76 GB of GPU memory on just five screenshots, nearly filling an 80 GB A100 or H100. Scale that to a real task with dozens of steps and you have a model that simply cannot run on any single accelerator. Running inference on UI-TARS-1.5-7B with bfloat16 and flash attention on a 5-screenshot example for a single step can take over 15 seconds on an H200 on average. Memory and latency are both walls.

Why existing compression methods fall short

Large Vision-Language Models have emerged as powerful engines for autonomous GUI agents, yet their deployment is severely constrained by the substantial memory footprint and latency of the Key-Value cache during long-horizon interactions. The research community has responded with a wave of KV compression techniques -- SnapKV, H2O, PyramidKV, GUIKV -- but these were largely designed for text-heavy LLMs or general vision tasks. They suffer from suboptimal performance in GUI scenarios due to a fundamental misalignment: unlike general visual tasks where attention sparsity varies across layers, GUI attention patterns exhibit uniform high-sparsity across all transformer layers.