Domain-Adaptive Full-Face Gaze Estimation via Novel-View-Synthesis and Feature Disentanglement

TL;DR

This paper introduces a novel training pipeline for cross-domain full-face gaze estimation, combining synthetic data generation via 3D reconstruction and an unsupervised domain adaptation method with feature disentanglement, improving performance across diverse domains.

Contribution

It presents a new data synthesis method using single-image 3D reconstruction and a disentangling autoencoder for unsupervised domain adaptation in gaze estimation.

Findings

Synthetic data with expanded head pose range performs comparably to real data.

The domain adaptation method enhances performance on multiple target domains.

The approach reduces the need for extensive real-world labeled data.

Abstract

Along with the recent development of deep neural networks, appearance-based gaze estimation has succeeded considerably when training and testing within the same domain. Compared to the within-domain task, the variance of different domains makes the cross-domain performance drop severely, preventing gaze estimation deployment in real-world applications. Among all the factors, ranges of head pose and gaze are believed to play significant roles in the final performance of gaze estimation, while collecting large ranges of data is expensive. This work proposes an effective model training pipeline consisting of a training data synthesis and a gaze estimation model for unsupervised domain adaptation. The proposed data synthesis leverages the single-image 3D reconstruction to expand the range of the head poses from the source domain without requiring a 3D facial shape dataset. To bridge the inevitable gap between synthetic and real images, we further propose an unsupervised domain adaptation method suitable for synthetic full-face data. We propose a disentangling autoencoder network to separate gaze-related features and introduce background augmentation consistency loss to utilize the characteristics of the synthetic source domain. Through comprehensive experiments, it shows that the model using only our synthetic training data can perform comparably to real data extended with a large label range. Our proposed domain adaptation approach further improves the performance on multiple target domains. The code and data will be available at https://github.com/ut-vision/AdaptiveGaze.