Is Tracking really more challenging in First Person Egocentric Vision?

TL;DR

This paper investigates whether the perceived difficulty of visual object tracking in egocentric vision is truly due to the first person perspective or other factors, using a new benchmark to isolate these variables.

Contribution

The authors introduce a benchmark that separates challenges of first person viewpoint from those of human-object activity understanding in egocentric vision.

Findings

Clarifies the sources of difficulty in egocentric tracking

Provides insights into the impact of viewpoint versus activity domain

Facilitates targeted improvements in egocentric vision methods

Abstract

Visual object tracking and segmentation are becoming fundamental tasks for understanding human activities in egocentric vision. Recent research has benchmarked state-of-the-art methods and concluded that first person egocentric vision presents challenges compared to previously studied domains. However, these claims are based on evaluations conducted across significantly different scenarios. Many of the challenging characteristics attributed to egocentric vision are also present in third person videos of human-object activities. This raises a critical question: how much of the observed performance drop stems from the unique first person viewpoint inherent to egocentric vision versus the domain of human-object activities? To address this question, we introduce a new benchmark study designed to disentangle such factors. Our evaluation strategy enables a more precise separation of challenges related to the first person perspective from those linked to the broader domain of human-object activity understanding. By doing so, we provide deeper insights into the true sources of difficulty in egocentric tracking and segmentation, facilitating more targeted advancements on this task.