Track, then Decide: Category-Agnostic Vision-based Multi-Object Tracking

TL;DR

This paper introduces a category-agnostic, model-free multi-object tracking method using segmentation masks, capable of tracking known and unknown objects without relying on specific detectors, suitable for complex environments.

Contribution

It proposes a novel segmentation mask-based tracker that can track both known and unknown objects, eliminating the need for category-specific detectors.

Findings

Achieves comparable performance to state-of-the-art methods on common categories

Can discover and track a wide variety of unknown objects

Maintains robust tracking in diverse environments

Abstract

The most common paradigm for vision-based multi-object tracking is tracking-by-detection, due to the availability of reliable detectors for several important object categories such as cars and pedestrians. However, future mobile systems will need a capability to cope with rich human-made environments, in which obtaining detectors for every possible object category would be infeasible. In this paper, we propose a model-free multi-object tracking approach that uses a category-agnostic image segmentation method to track objects. We present an efficient segmentation mask-based tracker which associates pixel-precise masks reported by the segmentation. Our approach can utilize semantic information whenever it is available for classifying objects at the track level, while retaining the capability to track generic unknown objects in the absence of such information. We demonstrate experimentally that our approach achieves performance comparable to state-of-the-art tracking-by-detection methods for popular object categories such as cars and pedestrians. Additionally, we show that the proposed method can discover and robustly track a large variety of other objects.