WristSonic: Enabling Fine-grained Hand-Face Interactions on Smartwatches Using Active Acoustic Sensing
Saif Mahmud, Kian Mahmoodi, Chi-Jung Lee, Francois Guimbretiere, Cheng, Zhang

TL;DR
WristSonic is a wrist-worn acoustic sensing system that accurately detects fine-grained hand-face interactions using ultrasonic reflections and a Transformer neural network, enabling privacy-conscious gesture recognition in daily settings.
Contribution
It introduces a novel ultrasonic sensing approach combined with deep learning for precise, low-power detection of diverse hand-face gestures on wearables.
Findings
Achieves 93.08% macro F1-score in controlled settings.
Achieves 82.65% macro F1-score in semi-in-the-wild settings.
Detects 21 distinct hand-face actions with high accuracy.
Abstract
Hand-face interactions play a key role in many everyday tasks, providing insights into user habits, behaviors, intentions, and expressions. However, existing wearable sensing systems often struggle to track these interactions in daily settings due to their reliance on multiple sensors or privacy-sensitive, vision-based approaches. To address these challenges, we propose WristSonic, a wrist-worn active acoustic sensing system that uses speakers and microphones to capture ultrasonic reflections from hand, arm, and face movements, enabling fine-grained detection of hand-face interactions with minimal intrusion. By transmitting and analyzing ultrasonic waves, WristSonic distinguishes a wide range of gestures, such as tapping the temple, brushing teeth, and nodding, using a Transformer-based neural network architecture. This approach achieves robust recognition of 21 distinct actions with a…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsInteractive and Immersive Displays · Tactile and Sensory Interactions · Gaze Tracking and Assistive Technology
