DARTH: Holistic Test-time Adaptation for Multiple Object Tracking

TL;DR

DARTH is a comprehensive test-time adaptation framework for multiple object tracking that enhances robustness to domain shifts by self-supervised detection adaptation and a novel patch contrastive loss for appearance modeling.

Contribution

It introduces a holistic test-time adaptation method for MOT, addressing domain shift challenges in detection and appearance representation.

Findings

Significant performance improvements across various domain shifts.

Effective self-supervised detection adaptation.

Novel patch contrastive loss enhances appearance modeling.

Abstract

Multiple object tracking (MOT) is a fundamental component of perception systems for autonomous driving, and its robustness to unseen conditions is a requirement to avoid life-critical failures. Despite the urge of safety in driving systems, no solution to the MOT adaptation problem to domain shift in test-time conditions has ever been proposed. However, the nature of a MOT system is manifold - requiring object detection and instance association - and adapting all its components is non-trivial. In this paper, we analyze the effect of domain shift on appearance-based trackers, and introduce DARTH, a holistic test-time adaptation framework for MOT. We propose a detection consistency formulation to adapt object detection in a self-supervised fashion, while adapting the instance appearance representations via our novel patch contrastive loss. We evaluate our method on a variety of domain shifts - including sim-to-real, outdoor-to-indoor, indoor-to-outdoor - and substantially improve the source model performance on all metrics. Code: https://github.com/mattiasegu/darth.