DIFAI: Diverse Facial Inpainting using StyleGAN Inversion

TL;DR

This paper introduces a novel diverse facial inpainting framework leveraging StyleGAN inversion, semantic component analysis, and region normalization to generate multiple plausible inpainted images, outperforming existing methods.

Contribution

The paper presents a new framework for diverse facial inpainting using StyleGAN inversion, semantic component identification, and a region normalization decoder, enhancing diversity and plausibility.

Findings

Outperforms several state-of-the-art inpainting methods

Generates multiple plausible inpainted images for the same masked input

Utilizes semantic components for more realistic inpainting results

Abstract

Image inpainting is an old problem in computer vision that restores occluded regions and completes damaged images. In the case of facial image inpainting, most of the methods generate only one result for each masked image, even though there are other reasonable possibilities. To prevent any potential biases and unnatural constraints stemming from generating only one image, we propose a novel framework for diverse facial inpainting exploiting the embedding space of StyleGAN. Our framework employs pSp encoder and SeFa algorithm to identify semantic components of the StyleGAN embeddings and feed them into our proposed SPARN decoder that adopts region normalization for plausible inpainting. We demonstrate that our proposed method outperforms several state-of-the-art methods.