Angle Range and Identity Similarity Enhanced Gaze and Head Redirection based on Synthetic data

TL;DR

This paper introduces a data augmentation technique using monocular 3D face reconstruction to enhance gaze and head redirection accuracy and realism in full-face images, especially at large angles.

Contribution

It presents a novel data augmentation method that extends redirection range and improves image quality and identity preservation using synthetic data.

Findings

Significant improvement in redirection angular accuracy.

Enhanced image quality and identity preservation.

Effective handling of large-angle redirection.

Abstract

In this paper, we propose a method for improving the angular accuracy and photo-reality of gaze and head redirection in full-face images. The problem with current models is that they cannot handle redirection at large angles, and this limitation mainly comes from the lack of training data. To resolve this problem, we create data augmentation by monocular 3D face reconstruction to extend the head pose and gaze range of the real data, which allows the model to handle a wider redirection range. In addition to the main focus on data augmentation, we also propose a framework with better image quality and identity preservation of unseen subjects even training with synthetic data. Experiments show that our method significantly improves redirection performance in terms of redirection angular accuracy while maintaining high image quality, especially when redirecting to large angles.