GHOST 2.0: generative high-fidelity one shot transfer of heads

TL;DR

GHOST 2.0 introduces a novel head swapping method that preserves identity and structure, seamlessly integrates heads into backgrounds, and outperforms existing techniques in complex scenarios.

Contribution

The paper presents two new modules for head reenactment and seamless integration, advancing head swapping technology with state-of-the-art results.

Findings

Outperforms baselines in head swapping tasks

Robust to extreme pose variations

Handles large hairstyle differences

Abstract

While the task of face swapping has recently gained attention in the research community, a related problem of head swapping remains largely unexplored. In addition to skin color transfer, head swap poses extra challenges, such as the need to preserve structural information of the whole head during synthesis and inpaint gaps between swapped head and background. In this paper, we address these concerns with GHOST 2.0, which consists of two problem-specific modules. First, we introduce enhanced Aligner model for head reenactment, which preserves identity information at multiple scales and is robust to extreme pose variations. Secondly, we use a Blender module that seamlessly integrates the reenacted head into the target background by transferring skin color and inpainting mismatched regions. Both modules outperform the baselines on the corresponding tasks, allowing to achieve state of the art results in head swapping. We also tackle complex cases, such as large difference in hair styles of source and target. Code is available at https://github.com/ai-forever/ghost-2.0