Analysis of the hands in egocentric vision: A survey

TL;DR

This survey reviews recent research on egocentric vision focusing on hand localization, activity understanding, and applications, highlighting datasets and approaches in the field.

Contribution

It categorizes existing methods for egocentric hand analysis and provides a comprehensive overview of datasets and techniques used.

Findings

Hand localization techniques have advanced significantly.

Various approaches exist for interpreting hand actions.

Multiple datasets support egocentric hand research.

Abstract

Egocentric vision (a.k.a. first-person vision - FPV) applications have thrived over the past few years, thanks to the availability of affordable wearable cameras and large annotated datasets. The position of the wearable camera (usually mounted on the head) allows recording exactly what the camera wearers have in front of them, in particular hands and manipulated objects. This intrinsic advantage enables the study of the hands from multiple perspectives: localizing hands and their parts within the images; understanding what actions and activities the hands are involved in; and developing human-computer interfaces that rely on hand gestures. In this survey, we review the literature that focuses on the hands using egocentric vision, categorizing the existing approaches into: localization (where are the hands or parts of them?); interpretation (what are the hands doing?); and application (e.g., systems that used egocentric hand cues for solving a specific problem). Moreover, a list of the most prominent datasets with hand-based annotations is provided.