Learning Unsupervised Gaze Representation via Eye Mask Driven Information Bottleneck

TL;DR

This paper introduces an unsupervised gaze representation learning framework that leverages eye masking and contrastive learning to improve gaze estimation accuracy without requiring gaze annotations.

Contribution

It proposes a novel self-supervised training scheme with eye masking and contrastive loss to extract gaze features from full-face images, reducing annotation dependence.

Findings

Outperforms existing unsupervised gaze estimation methods on multiple benchmarks.

Effectively captures gaze-related features without gaze annotations.

Enhances robustness and accuracy in gaze estimation tasks.

Abstract

Appearance-based supervised methods with full-face image input have made tremendous advances in recent gaze estimation tasks. However, intensive human annotation requirement inhibits current methods from achieving industrial level accuracy and robustness. Although current unsupervised pre-training frameworks have achieved success in many image recognition tasks, due to the deep coupling between facial and eye features, such frameworks are still deficient in extracting useful gaze features from full-face. To alleviate above limitations, this work proposes a novel unsupervised/self-supervised gaze pre-training framework, which forces the full-face branch to learn a low dimensional gaze embedding without gaze annotations, through collaborative feature contrast and squeeze modules. In the heart of this framework is an alternating eye-attended/unattended masking training scheme, which squeezes gaze-related information from full-face branch into an eye-masked auto-encoder through an injection bottleneck design that successfully encourages the model to pays more attention to gaze direction rather than facial textures only, while still adopting the eye self-reconstruction objective. In the same time, a novel eye/gaze-related information contrastive loss has been designed to further boost the learned representation by forcing the model to focus on eye-centered regions. Extensive experimental results on several gaze benchmarks demonstrate that the proposed scheme achieves superior performances over unsupervised state-of-the-art.