HorizonRelight Carries Lighting Across Video Diffusion Windows

Video diffusion backbones are usually trained on short clips. Relighting a longer scene therefore requires overlapping windows, each of which can estimate shadows, color temperature, and material response differently. Those small differences become visible jumps whenever a new window begins.

HorizonRelight, an ECCV 2026 paper from NVIDIA Research and USC’s Vision and Graphics Lab, addresses those boundary artifacts by carrying the relit state from one chunk into the next. Each window continues an established lighting trajectory instead of calculating the target appearance in isolation.

Where the Flicker Comes From

Long videos exceed the temporal range of most diffusion backbones, so inference systems divide them into manageable chunks. Overlapping frames can soften motion discontinuities, but they do not guarantee consistent illumination. Independent windows may interpret the same surface or light source differently, producing flicker and abrupt changes in appearance.

Passing Lighting State Forward

HorizonRelight frames the task as temporally conditioned latent domain translation. A latent is the model’s compressed internal representation of an image or video frame; the target domain is the relit output. The model passes target-domain latents from the end of one window into the beginning of the next, giving later chunks direct access to the lighting state already generated.

1. Establish the target: A relit first frame defines the desired illumination and appearance.
2. Process the first chunk: The diffusion model generates target-domain latents for the opening window.
3. Propagate the tail: Latents from the end of that window condition the next overlapping chunk.
4. Continue through the video: Every later window inherits state from its predecessor.

Training uses masked target-domain self-conditioning to make the propagated state useful. Portions of the relit context are hidden, and the model learns to reconstruct the missing information while continuing the sequence. This objective encourages the model to rely on cross-window context instead of treating it as optional input.

One Frame Sets the Lighting

The prompt-based workflow begins with a relit version of the first source frame. In the published pipeline, Google’s Nano Banana Pro image model generates that anchor from the requested lighting prompt. HorizonRelight then propagates the anchor’s illumination and appearance through the video, assigning the initial edit to an image model and temporal consistency to the video model.

Spatially stretched or misaligned anchor images can disrupt propagation. HorizonRelight addresses that case with a prompting-frame G-buffer, which provides per-pixel scene information that helps align the warm-start image with the source frame.

Four Workflows in the Paper

Workflow	Result
Prompt-based relighting	Applies requested illumination, including lighting derived from HDR environment maps.
First-frame editing	Propagates an edited opening frame across the full sequence.
Inverse decomposition	Estimates material properties alongside forward relighting.
Misaligned warm starts	Uses a prompting-frame G-buffer to handle spatial differences in the anchor image.

The reported experiments use long, in-the-wild videos rather than tightly controlled studio sequences. According to the paper, latent propagation reduces visible chunk-boundary artifacts and suppresses unintended appearance changes across windows.

A Pattern for Other Long-Video Models

Relighting supports production tasks such as matching foreground footage to a new background, changing a scene’s mood, and normalizing clips captured under different conditions. Diffusion models can produce strong individual frames, while temporal drift has limited their use on sequences that extend beyond the model’s training window.

Persistent target-domain state could also inform chunked style transfer, inpainting, and super-resolution systems. Each task faces a similar engineering problem when independently generated windows must preserve appearance and structure over time. HorizonRelight provides a concrete architecture for testing state propagation in those pipelines, although the paper’s evidence focuses on relighting.

Paper Live, Code Pending

The paper is available on arXiv, and the authors identify the work as an ECCV 2026 publication. The project page lists code as coming soon. No product pricing applies to the current research release.

The absence of a public implementation currently prevents developers from reproducing the official pipeline or measuring its runtime, memory requirements, and integration cost. The transferable design is already clear: preserve generated target-domain latents at each boundary and train the model to use that propagated state when continuing the next window.