iRadar: Synthesizing Millimeter-Waves from Wearable Inertial Inputs for Human Gesture Sensing

TL;DR

iRadar leverages wearable IMU data to synthesize mmWave radar signals for gesture recognition, enabling high-accuracy gesture detection without extensive radar datasets.

Contribution

The paper introduces a diffusion-based translation model and a transformer for cross-modal gesture recognition using wearable IMU data to synthesize radar signals.

Findings

Achieves 99.82% Top-3 accuracy across scenarios

Synthesizes radar signals from IMU data effectively

Reduces dependence on large-scale radar datasets

Abstract

Millimeter-wave (mmWave) radar-based gesture recognition is gaining attention as a key technology to enable intuitive human-machine interaction. Nevertheless, the significant challenge lies in obtaining large-scale, high-quality mmWave gesture datasets. To tackle this problem, we present iRadar, a novel cross-modal gesture recognition framework that employs Inertial Measurement Unit (IMU) data to synthesize the radar signals generated by the corresponding gestures. The key idea is to exploit the IMU signals, which are commonly available in contemporary wearable devices, to synthesize the radar signals that would be produced if the same gesture was performed in front of a mmWave radar. However, several technical obstacles must be overcome due to the differences between mmWave and IMU signals, the noisy gesture sensing of mmWave radar, and the dynamics of human gestures. Firstly, we develop a method for processing IMU and mmWave data to extract critical gesture features. Secondly, we propose a diffusion-based IMU-to-radar translation model that accurately transforms IMU data into mmWave data. Lastly, we devise a novel transformer model to enhance gesture recognition performance. We thoroughly evaluate iRadar, involving 18 gestures and 30 subjects in three scenarios, using five wearable devices. Experimental results demonstrate that iRadar consistently achieves 99.82% Top-3 accuracy across diverse scenarios.