Helios: An extremely low power event-based gesture recognition for always-on smart eyewear
Prarthana Bhattacharyya, Joshua Mitton, Ryan Page, Owen Morgan, Ben, Menzies, Gabriel Homewood, Kemi Jacobs, Paolo Baesso, David Trickett, Chris, Mair, Taru Muhonen, Rory Clark, Louis Berridge, Richard Vigars, Iain, Wallace

TL;DR
Helios is a low-power, real-time event-based gesture recognition system for smart eyewear, enabling intuitive interactions with minimal power consumption and high accuracy, suitable for always-on augmented reality devices.
Contribution
Helios introduces the first ultra-low-power event-based gesture recognition system optimized for smart eyewear, combining a tiny event camera with a CNN on a low-power platform.
Findings
91% gesture recognition accuracy
60ms real-time latency
20mW power consumption for camera
Abstract
This paper introduces Helios, the first extremely low-power, real-time, event-based hand gesture recognition system designed for all-day on smart eyewear. As augmented reality (AR) evolves, current smart glasses like the Meta Ray-Bans prioritize visual and wearable comfort at the expense of functionality. Existing human-machine interfaces (HMIs) in these devices, such as capacitive touch and voice controls, present limitations in ergonomics, privacy and power consumption. Helios addresses these challenges by leveraging natural hand interactions for a more intuitive and comfortable user experience. Our system utilizes a extremely low-power and compact 3mmx4mm/20mW event camera to perform natural hand-based gesture recognition for always-on smart eyewear. The camera's output is processed by a convolutional neural network (CNN) running on a NXP Nano UltraLite compute platform, consuming…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsGaze Tracking and Assistive Technology · Hand Gesture Recognition Systems · Biometric Identification and Security
MethodsALIGN
