Jiangnan University's MM-UAV Benchmark Tracks Drones With Three Fused Sensors
MM-UAV combines RGB, infrared, and event-camera data across 2.8 million frames to make drone tracking robust in the dark, in clutter, and at speed.
ProTakeaways
- MM-UAV releases 1,321 tri-modal sequences (RGB, IR, event) with 2.8M annotated frames across 30+ scenarios.
- First public benchmark combining event cameras with RGB and thermal for anti-UAV multi-object tracking.
- Baseline MMA-SORT adds offset-guided alignment and adaptive fusion modules on top of YOLOX.
- Event motion cues replace appearance embeddings for identity association between frames.
- Code, evaluation toolkit, and pretrained weights available under Apache-2.0 on GitHub.
- Dataset is ~400 GB, distributed via Baidu Cloud with Google Drive mirror coming.
Tracking small drones with a single camera falls apart the moment lighting drops, the background gets busy, or the target moves fast. A team from Jiangnan University and the University of Surrey wants to fix that with MM-UAV, the first large-scale benchmark that fuses three complementary sensors, along with a baseline tracker that shows how to actually use them together.
Single-modality UAV tracking often fails under low illumination, cluttered backgrounds, and rapid motion. Multi-modal tracking is more resilient, but progress has been throttled by the absence of dedicated public datasets. MM-UAV is the attempt to unblock that.
Three sensors, one synchronized stream
The dataset pairs standard RGB video with two sensors that see the world very differently:
- Thermal infrared (IR): measures heat, so it keeps working at night and against dark or camouflaged backgrounds.
- Event cameras: pixel-level sensors that fire asynchronously whenever brightness changes. They produce almost no output for static scenes but light up with microsecond-scale motion cues, which is ideal for fast, tiny drones.
- RGB: the usual appearance channel that everything else is calibrated against.

The scale is where things get interesting. The dataset spans over 30 challenging scenarios, with 1,321 synchronised multi-modal sequences and more than 2.8 million annotated frames. Sequences are split 1,200 for training and 121 for testing across the three modalities. The extracted archive is roughly 400 GB, which gives a sense of how much raw sensor data is packed in.