360VOT: A New Benchmark Dataset for Omnidirectional Visual Object Tracking

TL;DR

This paper introduces 360VOT, a large-scale benchmark dataset for omnidirectional visual object tracking, addressing unique challenges of 360-degree images and providing new evaluation metrics and baseline results.

Contribution

It presents a novel 360 tracking framework, a comprehensive dataset with diverse annotations, and tailored metrics for evaluating omnidirectional tracking performance.

Findings

20 state-of-the-art trackers evaluated on 360VOT

New metrics enable more accurate performance assessment

Baseline results establish a reference for future research

Abstract

360{\deg} images can provide an omnidirectional field of view which is important for stable and long-term scene perception. In this paper, we explore 360{\deg} images for visual object tracking and perceive new challenges caused by large distortion, stitching artifacts, and other unique attributes of 360{\deg} images. To alleviate these problems, we take advantage of novel representations of target localization, i.e., bounding field-of-view, and then introduce a general 360 tracking framework that can adopt typical trackers for omnidirectional tracking. More importantly, we propose a new large-scale omnidirectional tracking benchmark dataset, 360VOT, in order to facilitate future research. 360VOT contains 120 sequences with up to 113K high-resolution frames in equirectangular projection. The tracking targets cover 32 categories in diverse scenarios. Moreover, we provide 4 types of unbiased ground truth, including (rotated) bounding boxes and (rotated) bounding field-of-views, as well as new metrics tailored for 360{\deg} images which allow for the accurate evaluation of omnidirectional tracking performance. Finally, we extensively evaluated 20 state-of-the-art visual trackers and provided a new baseline for future comparisons. Homepage: https://360vot.hkustvgd.com