SLYKLatent: A Learning Framework for Gaze Estimation Using Deep Facial Feature Learning

TL;DR

SLYKLatent introduces a deep learning framework that significantly improves gaze estimation accuracy by combining self-supervised learning, patch-based networks, and innovative loss functions, effectively handling dataset uncertainties and domain shifts.

Contribution

The paper proposes a novel gaze estimation method that leverages self-supervised pretraining and a tri-branch network with an inverse variance loss to outperform existing approaches.

Findings

10.9% improvement on Gaze360

3.8% surpassing MPIIFaceGaze

11.6% better on ETH-XGaze subset

Abstract

In this research, we present SLYKLatent, a novel approach for enhancing gaze estimation by addressing appearance instability challenges in datasets due to aleatoric uncertainties, covariant shifts, and test domain generalization. SLYKLatent utilizes Self-Supervised Learning for initial training with facial expression datasets, followed by refinement with a patch-based tri-branch network and an inverse explained variance-weighted training loss function. Our evaluation on benchmark datasets achieves a 10.9% improvement on Gaze360, supersedes top MPIIFaceGaze results with 3.8%, and leads on a subset of ETH-XGaze by 11.6%, surpassing existing methods by significant margins. Adaptability tests on RAF-DB and Affectnet show 86.4% and 60.9% accuracies, respectively. Ablation studies confirm the effectiveness of SLYKLatent's novel components.