High-fidelity GAN Inversion with Padding Space

TL;DR

This paper introduces a novel GAN inversion method that incorporates padding space with spatial information, significantly enhancing image reconstruction quality and enabling more flexible, user-controlled image editing.

Contribution

It proposes involving the generator's padding space with instance-aware coefficients to improve inversion quality while maintaining the GAN's native manifold for versatile image editing.

Findings

Outperforms existing inversion methods in quality metrics

Enables flexible and detailed image editing

Maintains the integrity of the GAN's original manifold

Abstract

Inverting a Generative Adversarial Network (GAN) facilitates a wide range of image editing tasks using pre-trained generators. Existing methods typically employ the latent space of GANs as the inversion space yet observe the insufficient recovery of spatial details. In this work, we propose to involve the padding space of the generator to complement the latent space with spatial information. Concretely, we replace the constant padding (e.g., usually zeros) used in convolution layers with some instance-aware coefficients. In this way, the inductive bias assumed in the pre-trained model can be appropriately adapted to fit each individual image. Through learning a carefully designed encoder, we manage to improve the inversion quality both qualitatively and quantitatively, outperforming existing alternatives. We then demonstrate that such a space extension barely affects the native GAN manifold, hence we can still reuse the prior knowledge learned by GANs for various downstream applications. Beyond the editing tasks explored in prior arts, our approach allows a more flexible image manipulation, such as the separate control of face contour and facial details, and enables a novel editing manner where users can customize their own manipulations highly efficiently.