BLASt3R unifies visual SLAM and offline 3D reconstruction

BLASt3R, a paper from NAVER Labs Europe, applies one optimization pipeline to two traditionally separate workloads: visual SLAM from a live video stream and 3D reconstruction from an unordered image collection. The system uses the same optimizer and hyperparameters in both modes, while estimating camera calibration when intrinsics are unavailable.

Visual simultaneous localization and mapping, or visual SLAM, estimates a moving camera’s position while building a map of its surroundings. Offline structure from motion, or SfM, solves a related geometry problem after all images are available. Combining both workloads in one backend could reduce the code, tuning, and calibration requirements for robotics, augmented reality, and 3D content pipelines.

Why hybrid pipelines hit a wall

Classical SfM systems such as COLMAP detect keypoints, match them across images, and run bundle adjustment. This nonlinear optimization jointly refines camera poses, camera parameters, and 3D points by minimizing the distance between observed image features and their projected 3D locations.

Feed-forward models in the DUSt3R family instead use transformers to predict dense 3D pointmaps from images. Later hybrid systems pass neural correspondences or geometric priors into bundle adjustment, improving accuracy while retaining a conventional geometric objective.

Hybrid pipelines become expensive as the number of images grows because dense pairwise matching can require comparisons across many view pairs. An online system must also process each new frame within a limited compute and memory budget. BLASt3R addresses that bottleneck with a streaming, coarse-to-fine multi-view matcher.