Skip to main content
Back

NVIDIA's Fast-FoundationStereo Runs 10x Faster Without Losing Depth Accuracy

NVIDIA's Fast-FoundationStereo delivers zero-shot stereo depth estimation at real-time speeds with just 14.6M parameters, making foundation-model accuracy viable on edge hardware for the first time.

  • 10x faster than FoundationStereo: Fast-FoundationStereo runs in 49ms (21ms with TensorRT) vs 496ms, with only 14.6M parameters.
  • First real-time zero-shot stereo model: Achieves foundation-model-level generalization without per-domain fine-tuning, at real-time frame rates.
  • Three-part compression strategy: Knowledge distillation for the backbone, blockwise NAS for cost filtering, and structured pruning for the GRU refinement module.
  • Edge-ready: Peak memory of 0.63 GB fits NVIDIA Jetson Orin/Thor; supports TensorRT and ONNX export.
  • 1.4M pseudo-labeled real stereo pairs: Automatic data curation pipeline from internet stereo videos boosts zero-shot generalization for all methods trained on it.
  • Available now on Hugging Face and GitHub, integrated into NVIDIA TAO Toolkit under NVIDIA Open Model Agreement.

Stereo depth estimation has long been stuck at a frustrating fork in the road: you could have a model that generalizes well to new environments without retraining, or one that runs fast enough for real-world deployment. NVIDIA's new Fast-FoundationStereo is the first to credibly claim both at once. The model is now available on Hugging Face and integrated into NVIDIA's TAO Toolkit.

Stereo matching is the computer vision task of taking two images from side-by-side cameras and computing a disparity map -- a per-pixel estimate of how far left or right each point shifts between the two views, which directly encodes depth. It's the core sensing primitive for robotics, autonomous vehicles, and AR/VR. The problem is that accurate models have historically been either slow or brittle.

A field split in two

Stereo foundation models achieve strong zero-shot generalization but remain computationally prohibitive for real-time applications. Efficient stereo architectures, on the other hand, sacrifice robustness for speed and require costly per-domain fine-tuning. This meant that deploying a capable stereo model on a robot or autonomous vehicle required either accepting slow inference or retraining the model every time you changed environments -- neither of which is practical at scale.

To bridge this gap, NVIDIA presents Fast-FoundationStereo, a family of architectures that achieve, for the first time, strong zero-shot generalization at real-time frame rate. The resulting model can run over 10x faster than FoundationStereo while closely matching its zero-shot accuracy, thus establishing a new state-of-the-art among real-time methods.

Scatter plot comparing stereo vision algorithms by speed and generalization error, showing Fast-FoundationStereo dominating the accuracy-speed frontier

The divide-and-conquer approach

Rather than designing a fast model from scratch, the NVIDIA team started from the powerful but slow FoundationStereo and systematically attacked each of its three computational bottlenecks. They employed a divide-and-conquer acceleration strategy with three components: (1) knowledge distillation to compress the hybrid backbone into a single efficient student; (2) blockwise neural architecture search for automatically discovering optimal cost filtering designs under latency budgets, reducing search complexity exponentially; and (3) structured pruning for eliminating redundancy in the iterative refinement module.