Skip to main content
Back

MVAD Exposes How Sora 2 and Kling 3.0 Break Every Fake-Video Detector

A new 121,000-sample benchmark tests whether detectors can spot AI-generated video and audio together across 23 generators and three forgery patterns.

Pro

  • MVAD is the first general-purpose benchmark for detecting AI-generated video plus audio, covering 121,000 clips from 23 generators.
  • Three forgery patterns tested: fake-fake, fake-real (synthetic avatar with real voice), and real-fake (real video with fake audio).
  • Video-only detectors completely fail on real-fake samples, confirming multimodal detection is required for modern generative content.
  • Best cross-generator F1 is only 66.7% (LipFD), with severe drops when forgery patterns shift between train and test.
  • H.264 compression is the most damaging perturbation, threatening any detector deployed on social platforms.
  • Dataset and code available at github.com/HuMengXue0104/MVAD.

Video generators like Sora 2, Veo 3, and Kling 3.0 no longer stop at pretty pixels. They now output synchronized audio: ambient noise, sound effects, dialogue matched to lip movement. That capability breaks nearly every existing fake-video detector, because almost all of them were trained on the visual channel alone or on narrow facial-deepfake datasets. A new benchmark called MVAD tries to close that gap, and its early numbers suggest current detectors are in worse shape than most people assume.

The blind spot in fake-video research

Almost every prior AI-generated video dataset covers only pixels. The MVAD authors point out that most existing datasets contain only the visual modality, focusing on diversity and realism without generating synchronized audio, and the handful that include sound, like FakeAVCeleb, stay locked inside the facial-deepfake world. A detector trained only on visuals cannot flag a clip where the footage is genuine but the audio was synthesized to put fabricated words in someone's mouth, or where a real recording has been paired with a fabricated scene.

MVAD is billed as the first general large-scale dataset for Multimodal AI-Generated Video-Audio Detection. It covers four content categories (humans, animals, objects, scenes), two visual styles (photorealistic and anime), and three distinct forgery patterns rather than one.

Three ways to fake a clip

Each of the three forgery patterns breaks a different assumption baked into existing detectors: