VideoJev scores video answers directly

A video assistant may need only two timestamps for temporal grounding or one letter for multiple-choice QA, yet many multimodal large language models still decode answers one token at a time. The VideoJev project, developed by researchers from Nanjing University and collaborators, converts both tasks into candidate-scoring problems. It returns interval or option scores without an autoregressive decoding loop.

The researchers provide model weights, inference code, examples, and an accompanying paper. Their results show the largest latency gains when an agent asks several questions about the same clip, allowing VideoJev to reuse encoded video features or process a batch of queries together.

Why short answers still run long

Temporal grounding predicts when an event starts and ends within a video. Its common metric, mean intersection over union, or mIoU, measures the average overlap between predicted and reference intervals; higher values indicate closer alignment. Multiple-choice video QA selects one answer from a fixed set of textual options.

Conventional video MLLMs handle both tasks through text generation, giving them a flexible interface at the cost of sequential decoding. When the valid outputs are already constrained, every generated reasoning or answer token adds latency. The paper calls direct scoring a System One approach, borrowing Kahneman’s term for fast judgment; the concrete mechanism is a restricted output space with fixed score readouts.

One head ranks spans and answers

VideoJev retains a pretrained video MLLM as its backbone and adds a shared scoring head. For grounding, the head evaluates candidate temporal intervals against the video and query. For QA, it evaluates the supplied answer options. The highest-scoring candidate becomes the structured prediction, so downstream software receives timestamps or option scores directly.