UniMotion: Self-Supervised Learning for Cross-Domain IMU Motion Recognition

TL;DR

UniMotion is a self-supervised learning framework that enables cross-device and cross-population IMU gesture recognition with minimal labeled data, leveraging unlabeled human activity data for pre-training.

Contribution

It introduces a novel two-stage pipeline with token-based pre-training and text-guided fine-tuning for generalized IMU gesture recognition across diverse devices and users.

Findings

Achieves 85% accuracy with only 10% labeled data.

Outperforms existing self-supervised and specialized models.

Effective across hand and earbud gestures for blind and sighted users.

Abstract

IMU-based gesture interfaces are being increasingly adopted as efficient, accessible, and intuitive alternatives to traditional input methods, such as touchscreens and voice. However, current gesture recognition algorithms are tailored to work for specific devices (e.g., smartwatches vs. earbuds) or user populations (e.g., blind vs. sighted users), limiting their generalizability. In this paper, we design UniMotion, a generalized IMU-based gesture recognition framework that works across devices and populations with minimal training samples. To overcome the challenges and high cost of collecting large-scale labeled training data, UniMotion leverages readily available unlabeled human activity data. The UniMotion pipeline comprises two stages: (1) pre-training a motion representation model using abundant unlabeled human activity data, and (2) fine-tuning it with a small amount of labeled gesture data. For pre-training, we introduce a token-based strategy and embeddings that learn to identify and focus attention on the key motion signatures in the temporal data For fine-tuning, we design a text-guided classifier that can reliably differentiate between temporally or semantically similar gestures. We evaluate UniMotion across both hand gestures (captured through a smartwatch) and earbud gestures (captured through earbuds), using data collected from blind and sighted users. Across these diverse devices and user populations, UniMotion achieves an accuracy of 85\%, across an average of 13 gesture classes using only 10\% of labeled data for training. UniMotion significantly outperforms state-of-the-art self-supervised learning approaches and specialized gesture recognition models.