CUDA-GHR: Controllable Unsupervised Domain Adaptation for Gaze and Head Redirection

TL;DR

CUDA-GHR is an unsupervised domain adaptation framework that enables fine-grained control over gaze and head pose in generated images, improving estimation accuracy across domains.

Contribution

It introduces a novel method for disentangling visual attributes and adapting to new domains while maintaining attribute control in gaze and head pose generation.

Findings

Outperforms state-of-the-art methods in quantitative evaluations.

Generates high-quality images with controllable gaze and head pose.

Enhances downstream gaze and head pose estimation through synthetic data.

Abstract

The robustness of gaze and head pose estimation models is highly dependent on the amount of labeled data. Recently, generative modeling has shown excellent results in generating photo-realistic images, which can alleviate the need for annotations. However, adopting such generative models to new domains while maintaining their ability to provide fine-grained control over different image attributes, \eg, gaze and head pose directions, has been a challenging problem. This paper proposes CUDA-GHR, an unsupervised domain adaptation framework that enables fine-grained control over gaze and head pose directions while preserving the appearance-related factors of the person. Our framework simultaneously learns to adapt to new domains and disentangle visual attributes such as appearance, gaze direction, and head orientation by utilizing a label-rich source domain and an unlabeled target domain. Extensive experiments on the benchmarking datasets show that the proposed method can outperform state-of-the-art techniques on both quantitative and qualitative evaluations. Furthermore, we demonstrate the effectiveness of generated image-label pairs in the target domain for pretraining networks for the downstream task of gaze and head pose estimation. The source code and pre-trained models are available at https://github.com/jswati31/cuda-ghr.