Diverse Inpainting and Editing with GAN Inversion
Ahmet Burak Yildirim, Hamza Pehlivan, Bahri Batuhan Bilecen, Aysegul, Dundar

TL;DR
This paper introduces a novel method for inverting erased images into StyleGAN's latent space, enabling realistic and diverse inpainting and editing by combining features from erased images and random samples.
Contribution
The paper proposes a new encoder and mixing network architecture trained with a novel setup to improve inpainting diversity and realism for erased images in StyleGAN.
Findings
Significant improvement in inpainting quality over state-of-the-art methods
Enhanced diversity in generated inpaintings through latent code augmentation
Better color consistency between inpainted and original regions
Abstract
Recent inversion methods have shown that real images can be inverted into StyleGAN's latent space and numerous edits can be achieved on those images thanks to the semantically rich feature representations of well-trained GAN models. However, extensive research has also shown that image inversion is challenging due to the trade-off between high-fidelity reconstruction and editability. In this paper, we tackle an even more difficult task, inverting erased images into GAN's latent space for realistic inpaintings and editings. Furthermore, by augmenting inverted latent codes with different latent samples, we achieve diverse inpaintings. Specifically, we propose to learn an encoder and mixing network to combine encoded features from erased images with StyleGAN's mapped features from random samples. To encourage the mixing network to utilize both inputs, we train the networks with generated…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
Diverse Inpainting and Editing with GAN Inversion· youtube
Taxonomy
TopicsGenerative Adversarial Networks and Image Synthesis · Advanced Image and Video Retrieval Techniques · Advanced Vision and Imaging
MethodsInpainting
