AHS: Adaptive Head Synthesis via Synthetic Data Augmentations

TL;DR

AHS introduces a synthetic data augmentation method for adaptive head synthesis, enabling high-quality, robust portrait manipulation across diverse head poses, expressions, and hairstyles without paired training data.

Contribution

The paper presents a novel synthetic data augmentation strategy that improves head synthesis models' generalization to varied poses and expressions without needing paired datasets.

Findings

AHS outperforms existing methods in challenging real-world scenarios.

It maintains facial identity and expression fidelity across head variations.

AHS demonstrates robustness in preserving accessories during pose changes.

Abstract

Recent digital media advancements have created increasing demands for sophisticated portrait manipulation techniques, particularly head swapping, where one's head is seamlessly integrated with another's body. However, current approaches predominantly rely on face-centered cropped data with limited view angles, significantly restricting their real-world applicability. They struggle with diverse head expressions, varying hairstyles, and natural blending beyond facial regions. To address these limitations, we propose Adaptive Head Synthesis (AHS), which effectively handles full upper-body images with varied head poses and expressions. AHS incorporates a novel head reenacted synthetic data augmentation strategy to overcome self-supervised training constraints, enhancing generalization across diverse facial expressions and orientations without requiring paired training data. Comprehensive experiments demonstrate that AHS achieves superior performance in challenging real-world scenarios, producing visually coherent results that preserve identity and expression fidelity across various head orientations and hairstyles. Notably, AHS shows exceptional robustness in maintaining facial identity while drastic expression changes and faithfully preserving accessories while significant head pose variations.