FACT: Feature Adaptive Continual-learning Tracker for Multiple Object Tracking

TL;DR

The paper introduces FACT, a novel online continual-learning framework for multiple object tracking that leverages all past information to improve long-term occlusion handling while maintaining real-time performance.

Contribution

It proposes the FAC module and a two-stage association method, enabling real-time, feature-adaptive continual learning in MOT, which surpasses existing methods in long-term occlusion scenarios.

Findings

Achieves state-of-the-art online tracking results on MOT17 and MOT20 benchmarks.

Effectively utilizes all past tracking information for improved long-term occlusion handling.

Maintains real-time tracking speed despite incorporating continual learning.

Abstract

Multiple object tracking (MOT) involves identifying multiple targets and assigning them corresponding IDs within a video sequence, where occlusions are often encountered. Recent methods address occlusions using appearance cues through online learning techniques to improve adaptivity or offline learning techniques to utilize temporal information from videos. However, most existing online learning-based MOT methods are unable to learn from all past tracking information to improve adaptivity on long-term occlusions while maintaining real-time tracking speed. On the other hand, temporal information-based offline learning methods maintain a long-term memory to store past tracking information, but this approach restricts them to use only local past information during tracking. To address these challenges, we propose a new MOT framework called the Feature Adaptive Continual-learning Tracker (FACT), which enables real-time tracking and feature learning for targets by utilizing all past tracking information. We demonstrate that the framework can be integrated with various state-of-the-art feature-based trackers, thereby improving their tracking ability. Specifically, we develop the feature adaptive continual-learning (FAC) module, a neural network that can be trained online to learn features adaptively using all past tracking information during tracking. Moreover, we also introduce a two-stage association module specifically designed for the proposed continual learning-based tracking. Extensive experiment results demonstrate that the proposed method achieves state-of-the-art online tracking performance on MOT17 and MOT20 benchmarks. The code will be released upon acceptance.