Hard Occlusions in Visual Object Tracking

TL;DR

This paper highlights the persistent challenge of hard occlusions in visual object tracking, demonstrating that current state-of-the-art trackers struggle with these scenarios and perform inconsistently across different occlusion categories.

Contribution

The authors created a specialized dataset focusing on hard occlusion scenarios and evaluated recent trackers, revealing their limitations and the inadequacy of average performance scores for real-world assessment.

Findings

Hard occlusions remain a significant challenge for SOTA trackers.

Tracker performance varies greatly across different occlusion categories.

Average performance scores are insufficient to evaluate tracker robustness in real-world scenarios.

Abstract

Visual object tracking is among the hardest problems in computer vision, as trackers have to deal with many challenging circumstances such as illumination changes, fast motion, occlusion, among others. A tracker is assessed to be good or not based on its performance on the recent tracking datasets, e.g., VOT2019, and LaSOT. We argue that while the recent datasets contain large sets of annotated videos that to some extent provide a large bandwidth for training data, the hard scenarios such as occlusion and in-plane rotation are still underrepresented. For trackers to be brought closer to the real-world scenarios and deployed in safety-critical devices, even the rarest hard scenarios must be properly addressed. In this paper, we particularly focus on hard occlusion cases and benchmark the performance of recent state-of-the-art trackers (SOTA) on them. We created a small-scale dataset containing different categories within hard occlusions, on which the selected trackers are evaluated. Results show that hard occlusions remain a very challenging problem for SOTA trackers. Furthermore, it is observed that tracker performance varies wildly between different categories of hard occlusions, where a top-performing tracker on one category performs significantly worse on a different category. The varying nature of tracker performance based on specific categories suggests that the common tracker rankings using averaged single performance scores are not adequate to gauge tracker performance in real-world scenarios.