The Story in Your Eyes: An Individual-difference-aware Model for Cross-person Gaze Estimation

TL;DR

This paper introduces a person-specific gaze estimation model that explicitly accounts for individual differences, improving accuracy over existing methods by filtering invalid samples and calibrating predictions.

Contribution

The paper presents a novel person-aware gaze estimation framework with modules for validity assessment, self-calibration, and person-specific transformation, outperforming state-of-the-art methods.

Findings

Significant performance improvements on EVE, XGaze, and MPIIGaze datasets.

Won the GAZE 2021 Competition on the EVE dataset.

Effective handling of invalid samples like eye blinking images.

Abstract

We propose a novel method on refining cross-person gaze prediction task with eye/face images only by explicitly modelling the person-specific differences. Specifically, we first assume that we can obtain some initial gaze prediction results with existing method, which we refer to as InitNet, and then introduce three modules, the Validity Module (VM), Self-Calibration (SC) and Person-specific Transform (PT)) Module. By predicting the reliability of current eye/face images, our VM is able to identify invalid samples, e.g. eye blinking images, and reduce their effects in our modelling process. Our SC and PT module then learn to compensate for the differences on valid samples only. The former models the translation offsets by bridging the gap between initial predictions and dataset-wise distribution. And the later learns more general person-specific transformation by incorporating the information from existing initial predictions of the same person. We validate our ideas on three publicly available datasets, EVE, XGaze and MPIIGaze and demonstrate that our proposed method outperforms the SOTA methods significantly on all of them, e.g. respectively 21.7%, 36.0% and 32.9% relative performance improvements. We won the GAZE 2021 Competition on the EVE dataset. Our code can be found here https://github.com/bjj9/EVE_SCPT.