GazePrior: Zero-Shot AR/VR Eye Tracking via Learned 3D Gaze Reconstruction

TL;DR

GazePrior is a data-driven 3D gaze reconstruction model that enables zero-shot eye tracking in AR/VR by synthesizing realistic, diverse training data without additional data collection.

Contribution

Introducing GazePrior, a novel 3D gaze prior that allows zero-shot training of eye tracking models across different devices without extra data collection.

Findings

ET models trained with GazePrior data outperform previous zero-shot methods.

GazePrior achieves higher accuracy and robustness in eye tracking.

Synthesized data matches the realism and diversity of real data.

Abstract

Eye tracking (ET) is a foundational technology for advanced AR/VR applications. However, training ET models for every new ET device is challenging: real data collection is costly and time-consuming, while existing synthetic data generation methods lack realism. To remove the need for additional data collection while maintaining data quality, we introduce a data-driven 3D prior that models the distribution of human eyes across diverse identities, gaze directions, and light settings. This model, which we coin GazePrior, then enables sparse-input 3D reconstruction of annotated data collected with previous ET devices, which can in turn be rendered from the cameras of any target ET device. Our approach synthesizes data with the realism, diversity and ground-truth accuracy of real data collection without its prohibitive costs. Our experiments demonstrate that ET models trained with our synthesized data outperform previous zero-shot methods, achieving higher accuracy and robustness.