GazeShift: Unsupervised Gaze Estimation and Dataset for VR

TL;DR

GazeShift introduces an unsupervised, real-time gaze estimation framework tailored for VR headsets, leveraging a large-scale dataset and achieving high accuracy with minimal calibration and computational resources.

Contribution

The paper presents GazeShift, a novel unsupervised gaze estimation method specifically designed for near-eye VR imagery, along with the VRGaze dataset for training and evaluation.

Findings

Achieves 1.84° mean error with few-shot calibration.

Operates in 5 ms inference time on VR headset GPU.

Uses 10x fewer parameters and 35x fewer FLOPs than baseline methods.

Abstract

Gaze estimation is instrumental in modern virtual reality (VR) systems. Despite significant progress in remote-camera gaze estimation, VR gaze research remains constrained by data scarcity, particularly the lack of large-scale, accurately labeled datasets captured with the off-axis camera configurations typical of modern headsets. Gaze annotation is difficult since fixation on intended targets cannot be guaranteed. To address these challenges, we introduce VRGaze, the first large-scale off-axis gaze estimation dataset for VR, comprising 2.1 million near-eye infrared images collected from 68 participants. We further propose GazeShift, an attention-guided unsupervised framework for learning gaze representations without labeled data. Unlike prior redirection-based methods that rely on multi-view or 3D geometry, GazeShift is tailored to near-eye imagery, achieving effective gaze-appearance disentanglement in a compact, real-time model. GazeShift embeddings can be optionally adapted to individual users via lightweight few-shot calibration, achieving a 1.84{\deg} mean error on VRGaze. On the remote-camera MPIIGaze dataset, the model achieves a 7.15{\deg} person-agnostic error, doing so with 10x fewer parameters and 35x fewer FLOPs than baseline methods. Deployed natively on a VR headset GPU, inference takes only 5 ms. Combined with demonstrated robustness to illumination changes, these results highlight GazeShift as a label-efficient, real-time solution for VR gaze tracking. Project code and the VRGaze dataset are released at https://github.com/gazeshift3/gazeshift