HiFiGaze: Improving Eye Tracking Accuracy Using Screen Content Knowledge

TL;DR

This paper introduces HiFiGaze, a novel gaze estimation method that leverages high-quality cameras and screen content knowledge to improve accuracy on consumer devices, reducing tracking error by approximately 8%.

Contribution

It demonstrates how screen content information can be used to enhance gaze tracking accuracy, a novel approach in consumer device settings.

Findings

Mean tracking error reduced by ~8% with the new model.

Additional 10-20% accuracy improvement when camera is at the bottom of the device.

Effective segmentation of screen reflection enables robust gaze estimation.

Abstract

We present a new and accurate approach for gaze estimation on consumer computing devices. We take advantage of continued strides in the quality of user-facing cameras found in e.g., smartphones, laptops, and desktops - 4K or greater in high-end devices - such that it is now possible to capture the 2D reflection of a device's screen in the user's eyes. This alone is insufficient for accurate gaze tracking due to the near-infinite variety of screen content. Crucially, however, the device knows what is being displayed on its own screen - in this work, we show this information allows for robust segmentation of the reflection, the location and size of which encodes the user's screen-relative gaze target. We explore several strategies to leverage this useful signal, quantifying performance in a user study. Our best performing model reduces mean tracking error by ~8% compared to a baseline appearance-based model. A supplemental study reveals an additional 10-20% improvement if the gaze-tracking camera is located at the bottom of the device.