Is First Person Vision Challenging for Object Tracking?

TL;DR

This paper presents the first systematic analysis of object tracking in First Person Vision using a new benchmark dataset, revealing challenges and the need for further research to improve tracker performance in FPV tasks.

Contribution

It introduces TREK-150, a novel benchmark dataset, and provides an extensive evaluation of state-of-the-art trackers in FPV, highlighting current limitations and research gaps.

Findings

State-of-the-art trackers perform poorly in FPV scenarios.

TREK-150 dataset enables comprehensive evaluation of FPV tracking.

Further research is needed to improve tracking in FPV applications.

Abstract

Understanding human-object interactions is fundamental in First Person Vision (FPV). Tracking algorithms which follow the objects manipulated by the camera wearer can provide useful cues to effectively model such interactions. Despite a few previous attempts to exploit trackers in FPV applications, a methodical analysis of the performance of state-of-the-art visual trackers in this domain is still missing. In this short paper, we provide a recap of the first systematic study of object tracking in FPV. Our work extensively analyses the performance of recent and baseline FPV trackers with respect to different aspects. This is achieved through TREK-150, a novel benchmark dataset composed of 150 densely annotated video sequences. The results suggest that more research efforts should be devoted to this problem so that tracking could benefit FPV tasks. The full version of this paper is available at arXiv:2108.13665.