Whose hand is this? Person Identification from Egocentric Hand Gestures

TL;DR

This paper investigates whether the appearance and motion of egocentric hands can be used to identify individuals, exploring various visual cues and demonstrating potential benefits for gesture recognition tasks.

Contribution

It systematically studies egocentric hand identification using multiple visual cues and shows how this can enhance gesture recognition models.

Findings

Hand appearance can be distinctive enough for identification.

Different visual cues contribute variably to recognition accuracy.

Adversarial training can improve gesture recognition by reducing user-specific biases.

Abstract

Recognizing people by faces and other biometrics has been extensively studied in computer vision. But these techniques do not work for identifying the wearer of an egocentric (first-person) camera because that person rarely (if ever) appears in their own first-person view. But while one's own face is not frequently visible, their hands are: in fact, hands are among the most common objects in one's own field of view. It is thus natural to ask whether the appearance and motion patterns of people's hands are distinctive enough to recognize them. In this paper, we systematically study the possibility of Egocentric Hand Identification (EHI) with unconstrained egocentric hand gestures. We explore several different visual cues, including color, shape, skin texture, and depth maps to identify users' hands. Extensive ablation experiments are conducted to analyze the properties of hands that are most distinctive. Finally, we show that EHI can improve generalization of other tasks, such as gesture recognition, by training adversarially to encourage these models to ignore differences between users.