Skip to main content
Back

Tencent's VibeFlow Recolors and Relights Videos Without Paired Training Data

Researchers from Peking University and Tencent introduce a self-supervised framework that edits video color and lighting without paired training data.

Paper, Full Access
  • Peking University and Tencent researchers released VibeFlow, a self-supervised video color and light editor.
  • Trains without paired data by perturbing structure in the reference and color in the source video.
  • Introduces Residual Velocity Fields to correct discretization errors from flow-matching solvers during inference.
  • Built on Wan2.1-Fun-1.3B-Control with LoRA; only 8 hours on 8 A100 GPUs.
  • Beats RelightVid, TC-Light, ColorMNet, AnyV2V across VBench metrics on DAVIS-val.
  • Zero-shot handles relighting, recoloring, day-to-night, low-light enhancement, and object color editing.

Changing the lighting or colors of a video without breaking its structure or introducing flicker sounds simple until you try it. Existing solutions either require millions of paired training samples rendered in a virtual engine, or they lean on training-free tricks that produce inconsistent results. A team from Peking University and Tencent has published a paper introducing VibeFlow, a self-supervised approach that skips the paired data problem entirely by leveraging what pre-trained video generation models already know about light and color.

VibeFlow applications teaser showing relighting, recoloring, day-night translation, and low-light enhancement

The paired-data trap

Specialized approaches for color and light editing typically rely on task-specific synthetic datasets with supervised training. IC-Light, for instance, needed over 10 million images to finetune Stable Diffusion for image relighting. Video methods that follow this recipe often require laborious data synthesis and prohibitive training overhead, and depend on auxiliary rendering data or physical priors such as normal maps, albedo, and depth, which restricts them to narrow scenarios.

The authors frame the challenge as satisfying three objectives at once: temporal coherence, photorealistic color and light manipulation, and strict structural fidelity. Prior work usually trades off one of the three, and general video editors like AnyV2V or Wan2.1 tend to drift on object identity when asked to touch photometric properties.

Teaching a video model to disentangle by breaking things

The core intuition is that modern video generators already synthesize realistic footage with complex lighting, so they must implicitly understand these photometric properties. The team's job is to unlock that knowledge rather than teach it from scratch.