Detection of bimanual gestures everywhere: why it matters, what we need and what is missing

TL;DR

This paper develops and compares techniques for reliably detecting and classifying bimanual gestures in unconstrained environments using inertial data, achieving high accuracy in everyday activity recognition.

Contribution

It extends Gaussian Mixture Modelling and Regression methods for bimanual gesture detection, providing a robust approach for real-world applications.

Findings

Achieved 97% activity recognition accuracy

High recall rate of 100% for tested activities

Effective modeling of everyday bimanual gestures

Abstract

Bimanual gestures are of the utmost importance for the study of motor coordination in humans and in everyday activities. A reliable detection of bimanual gestures in unconstrained environments is fundamental for their clinical study and to assess common activities of daily living. This paper investigates techniques for a reliable, unconstrained detection and classification of bimanual gestures. It assumes the availability of inertial data originating from the two hands/arms, builds upon a previously developed technique for gesture modelling based on Gaussian Mixture Modelling (GMM) and Gaussian Mixture Regression (GMR), and compares different modelling and classification techniques, which are based on a number of assumptions inspired by literature about how bimanual gestures are represented and modelled in the brain. Experiments show results related to 5 everyday bimanual activities, which have been selected on the basis of three main parameters: (not) constraining the two hands by a physical tool, (not) requiring a specific sequence of single-hand gestures, being recursive (or not). In the best performing combination of modeling approach and classification technique, five out of five activities are recognized up to an accuracy of 97%, a precision of 82% and a level of recall of 100%.