Beyond Physical Labels: Redefining Domains for Robust WiFi-based Gesture Recognition

TL;DR

GesFi is a WiFi-based gesture recognition system that automatically identifies and aligns latent domain factors to improve robustness across different environments, significantly outperforming existing methods.

Contribution

Introduces WiFi latent domain mining and adversarial alignment techniques for robust cross-environment gesture recognition with commodity WiFi devices.

Findings

Achieves up to 78% performance improvement over baselines.

Effectively uncovers latent domains responsible for distribution shifts.

Demonstrates superior generalization across multiple datasets and environments.

Abstract

In this paper, we propose GesFi, a novel WiFi-based gesture recognition system that introduces WiFi latent domain mining to redefine domains directly from the data itself. GesFi first processes raw sensing data collected from WiFi receivers using CSI-ratio denoising, Short-Time Fast Fourier Transform, and visualization techniques to generate standardized input representations. It then employs class-wise adversarial learning to suppress gesture semantic and leverages unsupervised clustering to automatically uncover latent domain factors responsible for distributional shifts. These latent domains are then aligned through adversarial learning to support robust cross-domain generalization. Finally, the system is applied to the target environment for robust gesture inference. We deployed GesFi under both single-pair and multi-pair settings using commodity WiFi transceivers, and evaluated it across multiple public datasets and real-world environments. Compared to state-of-the-art baselines, GesFi achieves up to 78% and 50% performance improvements over existing adversarial methods, and consistently outperforms prior generalization approaches across most cross-domain tasks.