TinyGaze: Lightweight Gaze-Gesture Recognition on Commodity Mobile Devices

TL;DR

This paper introduces TinyGaze, a lightweight gaze-gesture recognition system for mobile devices, demonstrating high accuracy with minimal model size in a controlled pilot study.

Contribution

It presents an end-to-end pipeline using commodity ARKit data and a compact model, achieving high performance in gesture and user recognition tasks.

Findings

TinyHAR achieves Macro F1=0.960 for gesture recognition.

TinyHAR achieves Macro F1=0.997 for user identification.

Head pose dynamics are highly informative for gaze gestures.

Abstract

Gaze gestures can provide hands free input on mobile devices, but practical use requires (i) gestures users can learn and recall and (ii) recognition models that are efficient enough for on-device deployment. We present an end-to-end pipeline using commodity ARKit head/eye transforms and a scaffolded guidance-to-recall protocol grounded in learning theory. In a pilot feasibility study (N=4 participants; 240 trials; controlled single-session setting), we benchmark a compact time-series model (TinyHAR) against deeper baselines (DeepConvLSTM, SA-HAR) on 5-way gesture recognition and 4-way user identification. TinyHAR achieves strong performance in this pilot benchmark (Macro F1 = 0.960 for gesture recognition; Macro F1 = 0.997 for user identification) while using only 46k parameters. A modality analysis further indicates that head pose dynamics are highly informative for mobile gaze gestures, highlighting embodied head--eye coordination as a key design consideration. Although the small sample size and controlled setting limit generalizability, these results indicate a potential direction for further investigation into on-device gaze gesture recognition.