TripCEAiR: A Multi-Loss minimization approach for surface EMG based Airwriting Recognition

TL;DR

This paper introduces TripCEAiR, a multi-loss minimization framework for recognizing airwritten alphabets using surface EMG signals, achieving over 81% accuracy in user-dependent scenarios.

Contribution

It proposes a novel multi-loss framework combining triplet loss with classifier training for sEMG-based airwriting recognition, addressing a gap in dynamic gesture recognition research.

Findings

Achieved 81.26% accuracy in user-dependent scenarios.

Achieved 65.62% accuracy in user-independent scenarios.

Analyzed effects of triplet loss variations and embedding dimensions.

Abstract

Airwriting Recognition refers to the problem of identification of letters written in space with movement of the finger. It can be seen as a special case of dynamic gesture recognition wherein the set of gestures are letters in a particular language. Surface Electromyography (sEMG) is a non-invasive approach used to capture electrical signals generated as a result of contraction and relaxation of the muscles. sEMG has been widely adopted for gesture recognition applications. Unlike static gestures, dynamic gestures are user-friendly and can be used as a method for input with applications in Human Computer Interaction. There has been limited work in recognition of dynamic gestures such as airwriting, using sEMG signals and forms the core of the current work. In this work, a multi-loss minimization framework for sEMG based airwriting recognition is proposed. The proposed framework aims at learning a feature embedding vector that minimizes the triplet loss, while simultaneously learning the parameters of a classifier head to recognize corresponding alphabets. The proposed method is validated on a dataset recorded in the lab comprising of sEMG signals from 50 participants writing English uppercase alphabets. The effect of different variations of triplet loss, triplet mining strategies and feature embedding dimension is also presented. The best-achieved accuracy was 81.26% and 65.62% in user-dependent and independent scenarios respectively by using semihard positive and hard negative triplet mining. The code for our implementation will be made available at https://github.com/ayushayt/TripCEAiR.