HyperSSM Lets Objects Share Motion Clues to Beat Tracking Blindspots
HyperSSM fuses hypergraph reasoning with state space models to let correlated objects share motion cues, delivering state-of-the-art results across four MOT benchmarks.
ProTakeaways
- HyperSSM combines hypergraphs and state space models so correlated objects share motion cues during tracking
- Addresses two long-standing MOT problems: noisy learned predictions and trajectory loss during occlusion
- Hits SOTA on MOT17, MOT20, DanceTrack and SportsMOT benchmarks
- Achieves 78.5 HOTA on SportsMOT, beating DiffMOT by 2.3 points
- Ablations show 6-12 HOTA point gains over plain SSM and Kalman baselines on DanceTrack
- Code released at github.com/SkyeSong38/HyperMOT with 8-layer default config
Multi-object tracking has spent years wrestling with two stubborn failure modes: motion predictors that jitter under noise, and trajectories that fall apart the moment a target gets occluded. A new paper from Huazhong University of Science and Technology and the National University of Singapore proposes a different way of thinking about the problem. Rather than predicting each object's motion in isolation, HyperSSM lets objects with similar movement patterns reason together, using a hypergraph to share information and a state space model to keep trajectories temporally smooth.
The result is a tracker that posts state-of-the-art numbers on four different benchmarks, and the code is already on GitHub.
Why motion estimation keeps breaking
Motion prediction is the glue that holds identity together across frames in tracking-by-detection systems, and the field has been stuck between two imperfect options. Traditional trackers use hand-crafted Kalman filters that assume constant-velocity linear motion. They perform well in linear scenarios like surveillance video but deteriorate sharply in nonlinear cases such as dancing or sports. Learnable motion networks replace the Kalman filter with data-driven predictors that better capture complex nonlinear dynamics and substantially improve robustness, but their probabilistic outputs introduce fluctuations.
The authors frame this as two challenges. Learned predictors are noisy in easy linear scenes where a Kalman filter would be rock solid. And both approaches collapse when a target disappears behind another object, because there is simply no visual signal to work with.