UCF's VidTAG Pinpoints Every Video Frame on Earth With 25% More Accuracy
A new dual-encoder approach maps every frame of a video to precise GPS coordinates worldwide, beating GeoCLIP by 20% at the 1km mark.
PRO- VidTAG performs frame-to-GPS retrieval for video, producing per-frame coordinates and full trajectories at global scale.
- Dual encoder combines DINOv2 and CLIP features, aligned to a GeoCLIP-style GPS location encoder.
- TempGeo module enforces temporal consistency across frame embeddings during contrastive training.
- GeoRefiner uses cross-attention denoising to iteratively clean up an initial noisy trajectory.
- Beats fine-tuned GeoCLIP by ~20% at 1km on MSLS and ~25% on GAMa.
- Reaches ~95% city-level accuracy on CityGuessr68k, +25% over prior state-of-the-art.
Figuring out where a video was shot has long been either too coarse to be useful or too data-hungry to work at planetary scale. A new system called VidTAG, accepted at CVPR 2026, predicts a GPS coordinate for every frame of a video and stitches those predictions into a temporally consistent trajectory. The authors report gains of 20% over GeoCLIP at the one-kilometer threshold on Mapillary and roughly 25% on the CityGuessr68k global benchmark.
The work comes out of the University of Central Florida's Center for Research in Computer Vision (Mubarak Shah's group) with a collaborator at Luleå Tekniska Universitet, and it directly extends the line of research started by GeoCLIP.
Why coarse city labels weren't enough
Video geolocalization has two dominant recipes, and both hit a wall at global scale. Classification-based methods chop the Earth into a fixed grid of geographic cells and predict which cell a frame belongs to. Performance is limited by the predefined classes and often results in inaccurate localizations when an image's true position deviates from its class center. Fine if you only need to know a city, useless for placing a car on the right street.
Retrieval methods flip the problem: build a giant reference gallery of geotagged photos, then find the nearest match. Constructing a gallery large enough to cover the entire world is infeasible, so image-to-image approaches struggle at planetary scale.
GeoCLIP proposed the elegant workaround that VidTAG builds on: retrieve GPS coordinates directly instead of images. It uses a CLIP-inspired framework for image-to-GPS retrieval, jointly embedding pictures and geographic coordinates into a shared latent space. The location encoder transforms GPS coordinates into high-dimensional representations using Random Fourier Features combined with a hierarchical multi-resolution design. A gallery of latitude/longitude pairs is cheap; a gallery of Earth photos is not.
This story is for Pro members
You've reached the end of the free preview. Upgrade to AlphaSignal Pro to read the full article - and everything else behind the paywall.
Comments
