Realizing Fully-Integrated, Low-Power, Event-Based Pupil Tracking with Neuromorphic Hardware

TL;DR

This paper introduces a fully integrated, low-power, event-based pupil tracking system using neuromorphic hardware, enabling real-time, energy-efficient eye tracking suitable for wearable devices.

Contribution

It presents the first battery-powered, wearable pupil tracking system combining event-based sensing and neuromorphic processing on a commercial chip with novel neural network design.

Findings

Achieved 100 Hz binocular pupil tracking with under 5 mW power per eye.

Validated system on a new multi-user dataset.

Demonstrated practical, always-on eye tracking for wearable applications.

Abstract

Eye tracking is fundamental to numerous applications, yet achieving robust, high-frequency tracking with ultra-low power consumption remains challenging for wearable platforms. While event-based vision sensors offer microsecond resolution and sparse data streams, they have lacked fully integrated, low-power processing solutions capable of real-time inference. In this work, we present the first battery-powered, wearable pupil-center-tracking system with complete on-device integration, combining event-based sensing and neuromorphic processing on the commercially available Speck2f system-on-chip with lightweight coordinate decoding on a low-power microcontroller. Our solution features a novel uncertainty-quantifying spiking neural network with gated temporal decoding, optimized for strict memory and bandwidth constraints, complemented by systematic deployment mechanisms that bridge the reality gap. We validate our system on a new multi-user dataset and demonstrate a wearable prototype with dual neuromorphic devices achieving robust binocular pupil tracking at 100 Hz with an average power consumption below 5 mW per eye. Our work demonstrates that end-to-end neuromorphic computing enables practical, always-on eye tracking for next-generation energy-efficient wearable systems.