Transforming the Latent Space of StyleGAN for Real Face Editing

TL;DR

This paper introduces a new latent space, W++, for StyleGAN that improves real face editing by combining the benefits of existing spaces through transformer-based modifications, achieving better reconstruction and editing quality.

Contribution

Replacing fully-connected layers with transformers in StyleGAN's mapping network creates W++, a unified latent space that enhances face editing and reconstruction while maintaining high image quality.

Findings

W++ space outperforms W and W+ in face editing tasks

Maintains state-of-the-art generation quality with improved diversity

Supports existing inversion and editing algorithms with minimal changes

Abstract

Despite recent advances in semantic manipulation using StyleGAN, semantic editing of real faces remains challenging. The gap between the $W$ space and the $W$+ space demands an undesirable trade-off between reconstruction quality and editing quality. To solve this problem, we propose to expand the latent space by replacing fully-connected layers in the StyleGAN's mapping network with attention-based transformers. This simple and effective technique integrates the aforementioned two spaces and transforms them into one new latent space called $W$++. Our modified StyleGAN maintains the state-of-the-art generation quality of the original StyleGAN with moderately better diversity. But more importantly, the proposed $W$++ space achieves superior performance in both reconstruction quality and editing quality. Despite these significant advantages, our $W$++ space supports existing inversion algorithms and editing methods with only negligible modifications thanks to its structural similarity with the $W/W$+ space. Extensive experiments on the FFHQ dataset prove that our proposed $W$++ space is evidently more preferable than the previous $W/W$+ space for real face editing. The code is publicly available for research purposes at https://github.com/AnonSubm2021/TransStyleGAN.