CAMELTrack: Context-Aware Multi-cue ExpLoitation for Online Multi-Object Tracking

TL;DR

CAMELTrack introduces a data-driven, transformer-based association module for online multi-object tracking, outperforming heuristic-based methods while maintaining modularity and efficiency.

Contribution

The paper presents CAMEL, a novel transformer-based association module that learns complex cue interactions directly from data, replacing hand-crafted heuristics in tracking-by-detection.

Findings

Achieves state-of-the-art results on multiple benchmarks.

Maintains modularity and efficiency with off-the-shelf models.

Outperforms heuristic-based association methods.

Abstract

Online multi-object tracking has been recently dominated by tracking-by-detection (TbD) methods, where recent advances rely on increasingly sophisticated heuristics for tracklet representation, feature fusion, and multi-stage matching. The key strength of TbD lies in its modular design, enabling the integration of specialized off-the-shelf models like motion predictors and re-identification. However, the extensive usage of human-crafted rules for temporal associations makes these methods inherently limited in their ability to capture the complex interplay between various tracking cues. In this work, we introduce CAMEL, a novel association module for Context-Aware Multi-Cue ExpLoitation, that learns resilient association strategies directly from data, breaking free from hand-crafted heuristics while maintaining TbD's valuable modularity. At its core, CAMEL employs two transformer-based modules and relies on a novel association-centric training scheme to effectively model the complex interactions between tracked targets and their various association cues. Unlike end-to-end detection-by-tracking approaches, our method remains lightweight and fast to train while being able to leverage external off-the-shelf models. Our proposed online tracking pipeline, CAMELTrack, achieves state-of-the-art performance on multiple tracking benchmarks. Our code is available at https://github.com/TrackingLaboratory/CAMELTrack.