Skip to main content
Back

Symphony Beats Every Video AI Agent While Cutting Costs by 42%

A five-agent framework that decomposes long-video reasoning like human cognition beats prior systems by 5 points on LVBench while cutting inference cost.

Pro

  • Symphony is a five-agent system for long-video understanding proposed by Haiyang Yan and collaborators.
  • Hits state-of-the-art on LVBench (71.8), LongVideoBench, VideoMME Long, and MLVU.
  • Beats prior best on LVBench by 5 points while cutting per-query cost by 41.8% vs DVD.
  • Grounding agent switches between CLIP retrieval and VLM scoring based on question complexity.
  • Reflection agent acts as an independent verifier, adding 2.5 points on LVBench in ablations.
  • Training-free; code and prompts are open source, accepted at CVPR 2026.

A team led by Haiyang Yan at the Institute of Automation, Chinese Academy of Sciences, working with collaborators at Kuaishou Technology, has published a paper introducing Symphony, a multi-agent system that pushes long-form video understanding to state-of-the-art numbers on four major benchmarks. The system emulates how human cognition splits work between attention, perception, language, and reasoning, and it does so while spending less on API calls than the closest competitor.

Despite rapid progress with MLLM agents, they still struggle on long-form video tasks that combine dense information with extended temporal spans. Two failure modes hurt most. Simple task decomposition falls apart on long-chain reasoning, and shrinking the time context through embedding-based retrieval often discards evidence a complex question actually needs. Dumping an hour of footage into a single MLLM overwhelms it; pulling a handful of clips with CLIP throws away the rest.

One hard problem, five easier ones

Symphony borrows a structure from cognitive psychology and hands each dimension to a dedicated agent. Five specialists sit under a central planner:

  • Planning Agent: handles global task planning, agent scheduling, and answer generation.
  • Reflection Agent: evaluates the reasoning trajectory and, on detecting logical inconsistencies or thin evidence, writes critiques that trigger refinement.
  • Grounding Agent: identifies relevant video segments using either VLM-based scoring or CLIP retrieval, depending on query complexity.