Skip to main content
Back

Co-Fusion4D Beats BEVFusion by 5 Points on Self-Driving Perception

Co-Fusion4D tackles temporal feature drift in BEV-based 3D detectors, hitting 74.9% mAP and 75.6% NDS on nuScenes without test-time tricks

  • SOTA on nuScenes: Co-Fusion4D hits 74.9% mAP and 75.6% NDS without test-time augmentation or external data.
  • Core problem solved: Temporal BEV feature misalignment caused by moving objects and ego-motion across frames in multi-modal detectors.
  • Key design: Current-frame-centric strategy treats present data as dominant; historical frames only contribute after passing spatiotemporal filtering.
  • Dual Attention Fusion (DAF): Jointly models intra-frame spatial and inter-frame temporal attention, emphasizing motion-consistent regions.
  • Four-stage training: Decoupled curriculum freezes modality-specific components progressively to stabilize optimization across LiDAR, camera, and temporal domains.
  • No code yet: No public repository has been released alongside the paper.

Co-Fusion4D is a new multi-modal 3D object detection framework that takes direct aim at one of the most persistent headaches in autonomous driving perception: when a self-driving car moves through the world, its sensor data from one moment to the next doesn't line up cleanly. The result is blurry, inconsistent feature maps that make it harder to reliably detect objects. Co-Fusion4D proposes a principled fix, and the numbers back it up.

The Ghost in the Feature Map

Modern autonomous driving stacks rely heavily on Bird's-Eye View (BEV) representations -- a top-down grid that unifies data from LiDAR point clouds and multiple cameras into a single spatial canvas. The appeal is obvious: everything is in one coordinate system, and you can reason about the scene geometrically.

The problem is time. Object motion and ego-motion often induce cross-frame spatiotemporal inconsistencies in BEV-based detectors, leading to temporal BEV feature misalignment and degraded spatiotemporal consistency. Think of it this way: a car that was at position X in the last frame has moved by the time the current frame is captured. If you naively stack historical BEV features on top of current ones, the past car's "ghost" bleeds into the wrong location. Multiply this across dozens of moving objects and a moving ego-vehicle, and your feature map becomes a smeared mess.

Most existing approaches try to compensate with ego-motion warping -- rotating and translating past features to align with the current frame's coordinate system. Recent BEV-based detectors incorporate temporal information by fusing BEV features from previous timestamps, but the existing modeling process meets problems when dealing with complex and dynamic autonomous driving scenarios. Crucially, ego-motion correction only accounts for the car's own movement; independently moving objects like pedestrians and cyclists remain misaligned.

Current Frame as the North Star

Co-Fusion4D is a unified framework that explicitly preserves cross-frame spatiotemporal consistency and suppresses temporal feature drift. It adopts a current-frame-centric strategy, treating the current frame as the primary source of information while selectively incorporating historical frames after spatiotemporal filtering and alignment.