TrackingNet: A Large-Scale Dataset and Benchmark for Object Tracking in the Wild

TL;DR

TrackingNet introduces the first large-scale, diverse dataset and benchmark for object tracking in natural settings, enabling significant improvements and evaluation of deep learning trackers.

Contribution

It provides a comprehensive large-scale dataset with over 30,000 videos and a new benchmark for fair evaluation of object tracking algorithms.

Findings

Deep trackers fine-tuned on TrackingNet improve performance.

Evaluation of 20+ trackers shows tracking in the wild remains challenging.

Deep learning trackers benefit from large-scale diverse data.

Abstract

Despite the numerous developments in object tracking, further development of current tracking algorithms is limited by small and mostly saturated datasets. As a matter of fact, data-hungry trackers based on deep-learning currently rely on object detection datasets due to the scarcity of dedicated large-scale tracking datasets. In this work, we present TrackingNet, the first large-scale dataset and benchmark for object tracking in the wild. We provide more than 30K videos with more than 14 million dense bounding box annotations. Our dataset covers a wide selection of object classes in broad and diverse context. By releasing such a large-scale dataset, we expect deep trackers to further improve and generalize. In addition, we introduce a new benchmark composed of 500 novel videos, modeled with a distribution similar to our training dataset. By sequestering the annotation of the test set and providing an online evaluation server, we provide a fair benchmark for future development of object trackers. Deep trackers fine-tuned on a fraction of our dataset improve their performance by up to 1.6% on OTB100 and up to 1.7% on TrackingNet Test. We provide an extensive benchmark on TrackingNet by evaluating more than 20 trackers. Our results suggest that object tracking in the wild is far from being solved.