Gesture Recognition from body-Worn RFID under Missing Data

TL;DR

This paper presents a novel gesture recognition system using body-worn RFID tags, employing a graph-based neural network and data interpolation techniques, achieving high accuracy even with missing data.

Contribution

The work introduces a graph neural network approach combined with data imputation for robust gesture recognition from RFID signals, highlighting the importance of arm tags.

Findings

Achieved 98.13% accuracy for 21 gestures.

System maintains 89.28% accuracy in leave-one-person-out validation.

Arm tags are more critical than wrist tags for recognition.

Abstract

We explore hand-gesture recognition through the use of passive body-worn reflective tags. A data processing pipeline is proposed to address the issue of missing data. Specifically, missing information is recovered through linear and exponential interpolation and extrapolation. Furthermore, imputation and proximity-based inference are employed. We represent tags as nodes in a temporal graph, with edges formed based on correlations between received signal strength (RSS) and phase values across successive timestamps, and we train a graph-based convolutional neural network that exploits graph-based self-attention. The system outperforms state-of-the-art methods with an accuracy of 98.13% for the recognition of 21 gestures. We achieve 89.28% accuracy under leave-one-person-out cross-validation. We further investigate the contribution of various body locations on the recognition accuracy. Removing tags from the arms reduces accuracy by more than 10%, while removing the wrist tag only reduces accuracy by around 2%. Therefore, tag placements on the arms are more expressive for gesture recognition than on the wrist.