COOLer: Class-Incremental Learning for Appearance-Based Multiple Object Tracking

TL;DR

COOLer introduces a novel continual learning approach for appearance-based multiple object tracking that effectively mitigates catastrophic forgetting by combining contrastive learning with pseudo-labeling, enabling the model to adapt to new classes over time.

Contribution

This work presents COOLer, the first tracker to address class-incremental learning in MOT, incorporating contrastive learning and pseudo-labeling to retain past knowledge while learning new categories.

Findings

COOLer outperforms baseline methods on BDD100K and SHIFT datasets.

It effectively mitigates catastrophic forgetting in MOT tasks.

The proposed contrastive learning technique enhances instance representation disentanglement.

Abstract

Continual learning allows a model to learn multiple tasks sequentially while retaining the old knowledge without the training data of the preceding tasks. This paper extends the scope of continual learning research to class-incremental learning for multiple object tracking (MOT), which is desirable to accommodate the continuously evolving needs of autonomous systems. Previous solutions for continual learning of object detectors do not address the data association stage of appearance-based trackers, leading to catastrophic forgetting of previous classes' re-identification features. We introduce COOLer, a COntrastive- and cOntinual-Learning-based tracker, which incrementally learns to track new categories while preserving past knowledge by training on a combination of currently available ground truth labels and pseudo-labels generated by the past tracker. To further exacerbate the disentanglement of instance representations, we introduce a novel contrastive class-incremental instance representation learning technique. Finally, we propose a practical evaluation protocol for continual learning for MOT and conduct experiments on the BDD100K and SHIFT datasets. Experimental results demonstrate that COOLer continually learns while effectively addressing catastrophic forgetting of both tracking and detection. The code is available at https://github.com/BoSmallEar/COOLer.