Skip to main content
Back

HKUST's FreeScale Boosts 3D Scene Reconstruction by 2.7 dB Without Extra Captures

A CVPR 2026 paper turns sparse real-world captures into large-scale training data by sampling only the viewpoints a reconstructed scene can trust.

Pro

  • FreeScale turns sparse real captures into scalable NVS training data via certainty-aware view sampling.
  • A voxel certainty grid identifies reliable geometry using small, opaque Gaussians in a reconstructed 3DGS.
  • A view graph connects cameras by Weighted IoU of shared high-certainty visibility, not pose distance.
  • Adds 2.7 dB PSNR gain on LVSM for large camera motion out-of-distribution benchmarks.
  • Per-scene 3DGS with FreeScale beats DIFIX3D+ at roughly half the compute cost.
  • Code available at github.com/mvp-ai-lab/FreeScale; accepted at CVPR 2026.

Training generalizable novel view synthesis (NVS) models like LVSM has hit a familiar wall: there isn't enough real-world data with diverse, well-calibrated camera trajectories, and synthetic data from Blender-style pipelines carries an obvious domain gap. A team from HKUST, NUS, KTH, Surrey and Imperial College has proposed a data engine, FreeScale, that sidesteps the choice by treating an imperfect 3D reconstruction as a rich but noisy geometric proxy, then sampling from it only where the geometry is trustworthy.

Augmenting LVSM training with roughly 22% additional generated free-views improves sparse-view reconstruction from 18.75 to 21.45 PSNR, particularly for large camera motion. Code and a project page are already public.

Why naive sampling from a 3D reconstruction fails

The setup is that of most 3D vision pipelines. You have a scene captured as a handful of posed images, and you fit a 3D Gaussian Splatting (3DGS) representation, a set of anisotropic Gaussian blobs that can be re-rendered from any camera. In principle, you can now render arbitrary new views for free and use them as extra supervision.

Reconstructions from sparse inputs are riddled with floaters, ghosting and geometric noise, especially in regions with insufficient coverage or inaccurate camera poses. A random new camera pose is likely to point straight into these artifacts and produce garbage supervision. Previous scaling attempts either rely on 3D simulators that introduce a synthetic-to-real domain gap, or on diffusion models that generate photorealistic images but cannot supply the accurate camera poses NVS training needs.