Skip to main content
Back

University of Bonn's 3DTV Renders Unseen Camera Views at 40 FPS

A feedforward network from University of Bonn renders novel views at 40 FPS from just three cameras, using Delaunay triangulation and depth-guided fusion.

Pro

Paper, Full Access
  • 3DTV synthesizes novel views from three cameras at 40 FPS at 1024x1024 on an RTX 4090 with 2.2 GB memory
  • The paper combines Delaunay-based camera selection with a coarse-to-fine depth pyramid, no per-scene training
  • Uses a Ghost-based lightweight backbone with grouped correlation volumes and residual depth refinement
  • Beats GPS-Gaussian and ENeRF on most human-centric benchmarks; struggles on out-of-distribution LLFF scenes
  • TensorRT BF16 compilation drops inference from 117 ms to 24.5 ms per frame
  • Code and models will be released on acceptance; see the project page

Rendering a scene from a viewpoint no camera actually captured is one of those problems that keeps getting solved and unsolved. NeRF and Gaussian Splatting made it look photorealistic, at the cost of per-scene training runs that take minutes to hours. Fine for offline reconstruction, hopeless for live telepresence. A team at the University of Bonn is now proposing 3DTV, a feedforward network that produces novel views from only three input cameras at 40 FPS and 1024x1024 resolution, with no per-scene optimization required.

3DTV pairs a classical geometric idea (pick the right cameras using Delaunay triangulation) with a lightweight depth-guided neural pipeline. The authors don't claim to beat offline methods on raw quality; they claim to hold their own against other real-time sparse-view baselines while fitting in 2.2 GB of GPU memory after TensorRT compilation.

3DTV pipeline overview showing Delaunay triangulation, depth-guided synthesis, and coarse-to-fine refinement

The bandwidth vs. latency squeeze

Multi-camera capture rigs produce a lot of redundant data. Real-time streaming and rendering of multi-view video runs into a fundamental problem: raw capture generates enormous volumes, while only a small subset of views are actually needed to synthesize novel viewpoints. Existing systems trade off along a few axes. NeRF and Gaussian Splatting variants achieve photorealism but need per-scene training. Methods like ENeRF and GPS-Gaussian move toward real-time performance, but either need many cameras or degrade under wide baselines.

2-view approaches such as GPS-Gaussian/+ frequently suffer from depth ambiguity under wide baselines, leading to floating structures, duplicated geometry, and temporal instability. The core observation driving 3DTV is that not all views are equally valuable. Pick the geometrically right ones and you can throw the rest away.