Online Descriptor Enhancement via Self-Labelling Triplets for Visual Data Association

TL;DR

This paper introduces a self-supervised online descriptor refinement method for visual data association, enhancing multi-object tracking by adaptively improving descriptors during inference without extensive domain-specific training.

Contribution

It proposes a novel online self-labelling triplet-based training approach that refines deep descriptors in real-time, outperforming existing methods in object tracking tasks.

Findings

Achieves 94% reduction in parameters enabling online optimization.

Improves descriptor quality for multi-object tracking.

Surpasses other data association methods in tracking performance.

Abstract

Object-level data association is central to robotic applications such as tracking-by-detection and object-level simultaneous localization and mapping. While current learned visual data association methods outperform hand-crafted algorithms, many rely on large collections of domain-specific training examples that can be difficult to obtain without prior knowledge. Additionally, such methods often remain fixed during inference-time and do not harness observed information to better their performance. We propose a self-supervised method for incrementally refining visual descriptors to improve performance in the task of object-level visual data association. Our method optimizes deep descriptor generators online, by continuously training a widely available image classification network pre-trained with domain-independent data. We show that earlier layers in the network outperform later-stage layers for the data association task while also allowing for a 94% reduction in the number of parameters, enabling the online optimization. We show that self-labelling challenging triplets--choosing positive examples separated by large temporal distances and negative examples close in the descriptor space--improves the quality of the learned descriptors for the multi-object tracking task. Finally, we demonstrate that our approach surpasses other visual data-association methods applied to a tracking-by-detection task, and show that it provides better performance-gains when compared to other methods that attempt to adapt to observed information.