'Labelling the Gaps': A Weakly Supervised Automatic Eye Gaze Estimation

TL;DR

This paper introduces a weakly supervised gaze estimation method that significantly reduces the need for labeled data by leveraging limited annotations and knowledge distillation, enabling effective gaze tracking in unconstrained environments.

Contribution

The proposed approach effectively estimates gaze with minimal labeled data, using knowledge distillation from sparse annotations, and extends to scenarios with only start frame labels.

Findings

Reduces annotation effort to 2.67% of data

Maintains high performance with minimal labels

Validates on multiple benchmark datasets

Abstract

Over the past few years, there has been an increasing interest to interpret gaze direction in an unconstrained environment with limited supervision. Owing to data curation and annotation issues, replicating gaze estimation method to other platforms, such as unconstrained outdoor or AR/VR, might lead to significant drop in performance due to insufficient availability of accurately annotated data for model training. In this paper, we explore an interesting yet challenging problem of gaze estimation method with a limited amount of labelled data. The proposed method distills knowledge from the labelled subset with visual features; including identity-specific appearance, gaze trajectory consistency and motion features. Given a gaze trajectory, the method utilizes label information of only the start and the end frames of a gaze sequence. An extension of the proposed method further reduces the requirement of labelled frames to only the start frame with a minor drop in the generated label's quality. We evaluate the proposed method on four benchmark datasets (CAVE, TabletGaze, MPII and Gaze360) as well as web-crawled YouTube videos. Our proposed method reduces the annotation effort to as low as 2.67%, with minimal impact on performance; indicating the potential of our model enabling gaze estimation 'in-the-wild' setup.