NVIDIA's ArtiFixer Beats Prior 3D Scene Reconstruction by 3 dB
NVIDIA's ArtiFixer uses autoregressive diffusion to fill geometry gaps in 3D Gaussian Splatting scenes, beating prior SOTA by 1–3 dB PSNR
Takeaways
- ArtiFixer is an open autoregressive diffusion model from NVIDIA Research that fills missing geometry in 3D Gaussian Splatting scenes, presented at SIGGRAPH 2026.
- It beats all prior baselines by 1–3 dB PSNR on standard benchmarks, and outperforms next-best method GenFusion by nearly 3 dB on novel content generation.
- Key innovation: opacity mixing blends the degraded rendering with Gaussian noise based on the rendered opacity map, enabling consistency where geometry exists and full generative freedom where it doesn't.
- Built on Wan 2.1 14B, distilled into a causal autoregressive model that generates hundreds of frames in a single pass using Self-Forcing-style DMD distillation.
- Code, model weights, and training scripts are available on GitHub (Apache 2.0) and Hugging Face (NVIDIA research license).
- Three variants ship: ArtiFixer (direct rendering), ArtiFixer3D (distilled back into 3DGS), and ArtiFixer3D+ (both combined for best quality).
3D Gaussian Splatting (3DGS) is one of the most powerful tools available for real-time scene reconstruction and novel view synthesis. But it has a well-known blind spot: it only knows what the camera saw. Fly to an angle the camera never visited, and the reconstruction collapses into floaters, holes, and visual noise. NVIDIA Research's ArtiFixer is a new open model that directly targets this failure mode, and the results are striking.
The problem no one had cleanly solved
Per-scene optimization methods such as 3D Gaussian Splatting provide state-of-the-art novel view synthesis quality but extrapolate poorly to under-observed areas. Prior work tried to fix this with generative models, but ran into two hard walls. Methods that leverage generative priors to correct artifacts hold promise but currently suffer from two shortcomings: scalability, as existing methods use image diffusion models or bidirectional video models that are limited in the number of views they can generate in a single pass, requiring a costly iterative distillation process for consistency. The second problem was quality: generators trained on corrupted renderings tend to freeze up in completely unobserved regions where every input pixel is black, a failure mode called mode collapse.
The core tension is between consistency (respecting what the camera did capture) and generative capacity (hallucinating plausible content where it didn't). Previous methods could do one or the other, but not both at the same time.
Two clever ideas, stacked
ArtiFixer solves this with a two-stage pipeline built on top of Wan 2.1 T2V-14B, a 14-billion-parameter open video diffusion model. The two key innovations are:
- Opacity mixing: Instead of starting generation from pure Gaussian noise (which loses scene context) or from the degraded rendering directly (which collapses in empty regions), ArtiFixer blends the two based on the rendered opacity map. Where the scene has geometry, it starts from the existing rendering. Where there's nothing, it starts from noise. This gives the model scene fidelity where data exists and full generative freedom where it doesn't.
- Causal autoregressive distillation: The bidirectional teacher model is distilled into a causal autoregressive model using Self-Forcing-style DMD (Distribution Matching Distillation). The related source code provides implementations for training, evaluation, and inference, supporting various stages including bidirectional training, diffusion forcing, and Self-Forcing-style DMD distillation. The resulting model generates hundreds of frames in a single forward pass, rather than needing to iterate frame-by-frame.
The opacity mixing formula is elegant in its simplicity. For each frame, the model receives a mixed latent:
# z_mix blends the degraded rendering with Gaussian noise
# based on the rendered opacity map O_z
# Where opacity is high: use the existing rendering
# Where opacity is zero (unseen): use pure noise
z_mix = O_z * z_degraded + (1 - O_z) * epsilon
This single equation is what unlocks both consistency and generative capability simultaneously.
Three variants, one pipeline
The team ships three variants with different quality/speed tradeoffs:
- ArtiFixer: Directly renders novel views from the autoregressive generator. Sharpest output, fastest to run.
- ArtiFixer3D: Distills the generator's outputs back into the underlying 3DGS representation. More consistent with source images, slightly blurrier due to the explicit 3D representation.
- ArtiFixer3D+: Runs the autoregressive model again on top of ArtiFixer3D. Restores sharpness while keeping high consistency with the original captures.
The numbers are hard to ignore
Per-scene methods like 3DGS look great on captured views but collapse off-trajectory. ArtiFixer repairs those artifacts and beats prior SOTA by 1–3 dB PSNR. On the challenging 3-view split of MipNeRF 360, ArtiFixer exceeds, to its knowledge, all other previously published work, including previous SOTA method CAT3D.
The gap is especially dramatic on the novel content generation benchmark, where scenes are reconstructed with large unobserved regions. ArtiFixer outperforms the next-best method (GenFusion) by almost 3 dB in PSNR. For context, a 1 dB improvement in PSNR is typically considered significant in the field. Here's a summary of results on the Nerfbusters and DL3DV benchmarks:
| Method | Nerfbusters PSNR | DL3DV PSNR |
|---|---|---|
| 3DGS (baseline) | 17.66 | 17.18 |
| Difix3D+ (prior SOTA) | 18.51 | 17.99 |
| ArtiFixer | 19.83 | 19.73 |
| ArtiFixer3D | 20.24 | 20.14 |
| ArtiFixer3D+ | 20.12 | 20.06 |
What it's good at, and where it struggles
ArtiFixer is the only method that can generate plausible visuals in unobserved areas while respecting source fidelity. It also retains enough generative capability that it can produce output from text prompts alone or from reference views without any initial rendering, making it flexible beyond its primary use case.
The limitations are honest. The paper notes that inference speed, while reaching interactive rates, is still significantly slower than direct rendering from neural scene representations. The model also decodes frames in temporal chunks, which can introduce latency in real-time applications like embodied AI. And in completely empty regions, fewer denoising steps can hurt sharpness and temporal consistency.
How it was built
ArtiFixer was developed by NVIDIA's Spatial Intelligence Lab and is based on Wan2.1's 14B model. Training used 128 H100 GPUs with a batch size of 128. The process has three stages: supervised finetuning on reconstruction-conditioned video clips, diffusion forcing finetuning to prepare for autoregressive rollout, and finally DMD distillation to produce a fast few-step generator. Training data was generated from the DL3DV-10K dataset using a camera selection strategy that deliberately creates large empty regions the model must learn to inpaint.
What this means for the field
The dominant assumption in 3D reconstruction has been that generative models and explicit 3D representations are separate tools. You either reconstruct a scene or you generate it. ArtiFixer challenges that framing directly: the 3D reconstruction acts as a strong conditioning signal that prevents hallucination and drift, while the generative model fills in what the reconstruction cannot. The paper claims this is the first work to explore the intersection of explicit 3D reconstruction and autoregressive video generation.
The practical implications are significant for anyone building pipelines that require free-viewpoint navigation of real scenes:
- VR/AR content creation: Captured environments can now be navigated freely without visible holes or floaters.
- Autonomous driving simulation: Sparse sensor captures can be extended to cover viewpoints the vehicle never visited.
- Digital twins: Incomplete scans of physical spaces can be completed plausibly and consistently.
- Film and VFX: Camera trajectories can be extended beyond what was physically filmed.
Getting started
The ArtiFixer code and model weights are now released. The model is available on Hugging Face and the full codebase, including training scripts, is on GitHub. The repo is released under Apache 2.0. The model is for research and development only, and use of the model is governed by the NVIDIA License. The base Wan 2.1 model weights are Apache 2.0 licensed.
To run inference on your own COLMAP scene, the workflow is roughly: prepare your scene with data_processing.prepare_colmap_artifixer_inputs, download the artifixer-14b.pt checkpoint from Hugging Face, and run model_eval.run_inference. The full pipeline, including ArtiFixer3D distillation back into a 3DGS representation, is documented in the paper. The model requires a CUDA environment and is distributed as a Docker image for reproducibility.
The community response has been warm. Users praised NVIDIA's release of ArtiFixer code and weights to fix 3DGS artifacts because it addresses issues they've struggled with and seems effective enough to try immediately. Given how common the off-trajectory artifact problem is in production 3D pipelines, that reaction makes sense.