Translating Signals to Languages for sEMG-Based Activity Recognition

TL;DR

This paper introduces LLM-sEMG, a novel framework that leverages large language models to interpret sEMG signals for activity recognition, achieving high accuracy by translating signals into a language-like form.

Contribution

The paper proposes a new approach that uses large language models as sEMG activity recognizers through a language-oriented mapping mechanism.

Findings

Achieves high accuracy in sEMG activity recognition

Demonstrates effective signal-to-language mapping strategies

Validates the framework with extensive experiments

Abstract

Surface electromyography (sEMG) signal-based activity recognition has attracted increasing research attention in recent years. To develop accurate sEMG signal-based activity recognizers, numerous approaches have been proposed. Some studies focus on designing larger and more expressive model architectures to enhance the representational capacity of sEMG signals, while others aim to enrich model priors through large-scale pretraining, thereby improving recognition performance. Recently, large language models (LLMs) have shown remarkable generalization and reasoning capabilities in natural language processing, whose implicit knowledge, learned from extensive linguistic descriptions of actions, opens new possibilities for interpreting sEMG signals and inferring activity intentions. Motivated by this, we propose LLM-sEMG, a novel framework that leverages LLMs as sEMG activity recognizers. Within this framework, we design a language-oriented mapping mechanism that converts continuous sEMG sequences into sEMG language, integrating several strategies to further facilitate the signal-to-language mapping process. Extensive experiments demonstrate that the proposed framework achieves highly accurate sEMG signal-based activity recognition using large language models.