---
title: "StreamEval Opens the First Benchmark Testing Video AI Mid-Session"
subtitle: "StreamEval scores interactive streaming video models across five tasks, with incremental inputs, 15 to 300 second sessions, and an open harness."
url: "https://alphasignal.ai/news/streameval-opens-the-first-benchmark-testing-video-ai-mid-session"
author: "AlphaSignal Newsroom"
author_url: "https://alphasignal.ai/editorial-team"
company: "AlphaSignal"
company_url: "https://alphasignal.ai/company/alphasignal"
published: "2026-10-25T10:35:22.000Z"
updated: "2026-11-04T06:17:39.649Z"
type: "Repo"
topics:
  - "Benchmarks"
  - "Video"
subtopics:
  - "Video Generation"
  - "Video Editing"
  - "Video Understanding"
upvotes: 220
source_url: "https://github.com/StreamEval/StreamEval"
comments: 0
truncated: true
---

# StreamEval Opens the First Benchmark Testing Video AI Mid-Session

> StreamEval scores interactive streaming video models across five tasks, with incremental inputs, 15 to 300 second sessions, and an open harness.

Most video-generation benchmarks score a finished clip against one prompt, leaving them unable to measure systems that receive instructions while rendering. [StreamEval](https://github.com/StreamEval/StreamEval) targets those systems with five interactive tasks lasting 15 to 300 seconds. Its open release includes benchmark data, a [live leaderboard](https://streameval.ai/), and a reproducible evaluation harness.

## Five workflows, no catch-all rank

During each session, text, speech, images, source video, or camera controls arrive incrementally along a timeline. The harness records generated output and timestamps, then sends the completed session to an automated judge. Scores remain separate by task, allowing developers to distinguish avatar, world-generation, and editing performance.

__StreamEval tasks, inputs, and scoring dimensions__
| Task                   | Inputs                                                     | Metrics                                                   |
| ---------------------- | ---------------------------------------------------------- | --------------------------------------------------------- |
| Digital-human dialogue | Portrait, persona, and user speech or text turns           | Quality, A-V Sync, Stability, Dialogue, Identity          |
| Avatar performance     | Portrait, speech to perform, and live directions           | Quality, A-V Sync, Stability, Following, Identity         |
| World generation       | Opening prompt or image, live prompts, and camera controls | Quality, Stability, Following, Continuity                 |
| Instruction editing    | Live source video and text editing instructions            | Quality, Stability, Following, Edit Success, Preservation |

_Preview only. Full text requires an AlphaSignal Pro account._

## Sources

- [StreamEval GitHub Repository](https://github.com/StreamEval/StreamEval)
- [StreamEval Leaderboard](https://streameval.ai/)
- [StreamEval Paper (arXiv)](https://arxiv.org/pdf/2610.28402)
- [StreamEval Dataset on Hugging Face](https://huggingface.co/datasets/StreamEval/StreamEval)
