CBA Matches People Across Day and Night Cameras Without Human Labels
A new framework brings causal inference to unsupervised cross-modality person re-identification, closing the labeling gap for 24/7 surveillance video.
PRO- New framework Causal Bootstrapped Alignment (CBA) tackles unsupervised video visible-infrared person re-identification without cross-modality labels.
- Uses structural causal model to intervene on modality and temporal confounders before clustering pseudo-labels are generated.
- Modality-Perturbation Bootstrapping swaps color/thermal statistics via AdaIN; Temporal-Topology Bootstrapping shuffles adjacent frames to break motion shortcuts.
- Prototype-Guided Uncertainty Refinement uses visible clusters as anchors and applies soft supervision to ambiguous infrared matches.
- Beats prior best unsupervised baseline (PGM) by +20.2 Rank-1 on HITSZ-VCM I2V and +22.6 on V2I.
- Built on CLIP ViT-B/16 with VLD spatio-temporal prompting backbone; trains in three stages over 120 epochs.
Matching a person filmed on a daytime color camera against footage from a nighttime infrared camera is one of the messier problems in computer vision. Existing methods that do this well require humans to painstakingly label who is who across two very different-looking modalities. A new paper proposes an unsupervised approach that skips those labels entirely, borrowing tools from causal inference.
The framework is called Causal Bootstrapped Alignment (CBA), and it targets a task the authors call USL-VVI-ReID: unsupervised learning for video-based visible-infrared person re-identification. On the HITSZ-VCM and BUPTCampus benchmarks, CBA significantly outperforms existing unsupervised image-based visible-infrared methods when extended to the video setting, establishing state-of-the-art performance on this task.
Why the naive recipe collapses
The obvious approach would be: take an off-the-shelf pretrained encoder like CLIP, run clustering on visible tracklets, run clustering on infrared tracklets, then match the clusters across modalities. In practice this falls apart. The authors identify two intertwined failure modes that appear as soon as you point a generic encoder at cross-modality video.
- Identity confusion within a modality. Different people get tangled together in the embedding space, and the visible-infrared gap ends up larger than the gap between different identities.
- Granularity mismatch across modalities. Visible video produces many fine clusters; infrared video produces far fewer, coarser ones. Bipartite matching then yields ambiguous one-to-many correspondences instead of clean identity pairs.
This story is for Pro members
You've reached the end of the free preview. Upgrade to AlphaSignal Pro to read the full article - and everything else behind the paywall.
Comments
