iFace: Hand-Over-Face Gesture Recognition Leveraging Impedance Sensing

TL;DR

iFace is a wearable impedance-sensing system that unobtrusively recognizes hand-over-face gestures via shoulder electrodes, enabling implicit interaction detection during conversations without sensors on the face or hands.

Contribution

This work introduces a novel shoulder-based impedance sensing configuration for gesture recognition, avoiding direct contact with the face or hands.

Findings

Achieved 82.58% macro F1 score in gesture classification

Successfully recognized six different hand-over-face gestures

Demonstrated potential for implicit interaction in communication scenarios

Abstract

Hand-over-face gestures can provide important implicit interactions during conversations, such as frustration or excitement. However, in situations where interlocutors are not visible, such as phone calls or textual communication, the potential meaning contained in the hand-over-face gestures is lost. In this work, we present iFace, an unobtrusive, wearable impedance-sensing solution for recognizing different hand-over-face gestures. In contrast to most existing works, iFace does not require the placement of sensors on the user's face or hands. Instead, we proposed a novel sensing configuration, the shoulders, which remains invisible to both the user and outside observers. The system can monitor the shoulder-to-shoulder impedance variation caused by gestures through electrodes attached to each shoulder. We evaluated iFace in a user study with eight participants, collecting six kinds of hand-over-face gestures with different meanings. Using a convolutional neural network and a user-dependent classification, iFace reaches 82.58 \% macro F1 score. We discuss potential application scenarios of iFace as an implicit interaction interface.