AM-SORT: Adaptable Motion Predictor with Historical Trajectory Embedding for Multi-Object Tracking

TL;DR

AM-SORT introduces an adaptable transformer-based motion predictor with historical trajectory embedding for multi-object tracking, effectively handling non-linear motion and occlusions, outperforming traditional Kalman Filter-based methods.

Contribution

This paper presents a novel transformer-based motion predictor with historical trajectory embedding, replacing Kalman Filters in SORT-series trackers to better estimate non-linear object movements.

Findings

Achieves 56.3 IDF1 and 55.6 HOTA on DanceTrack

Effectively predicts non-linear movements under occlusions

Outperforms traditional Kalman Filter-based trackers

Abstract

Many multi-object tracking (MOT) approaches, which employ the Kalman Filter as a motion predictor, assume constant velocity and Gaussian-distributed filtering noises. These assumptions render the Kalman Filter-based trackers effective in linear motion scenarios. However, these linear assumptions serve as a key limitation when estimating future object locations within scenarios involving non-linear motion and occlusions. To address this issue, we propose a motion-based MOT approach with an adaptable motion predictor, called AM-SORT, which adapts to estimate non-linear uncertainties. AM-SORT is a novel extension of the SORT-series trackers that supersedes the Kalman Filter with the transformer architecture as a motion predictor. We introduce a historical trajectory embedding that empowers the transformer to extract spatio-temporal features from a sequence of bounding boxes. AM-SORT achieves competitive performance compared to state-of-the-art trackers on DanceTrack, with 56.3 IDF1 and 55.6 HOTA. We conduct extensive experiments to demonstrate the effectiveness of our method in predicting non-linear movement under occlusions.