Self-Distilled StyleGAN: Towards Generation from Internet Photos

TL;DR

This paper introduces a self-distillation method for StyleGAN that enables high-quality image generation from uncurated Internet photos by filtering outliers and handling multi-modal data distributions.

Contribution

It proposes a novel self-distillation approach with dataset filtering and perceptual clustering to adapt StyleGAN for raw, diverse Internet images.

Findings

Effective removal of outliers improves image quality.

Clustering enhances diversity preservation in generated images.

Method demonstrates success on diverse Internet-collected datasets.

Abstract

StyleGAN is known to produce high-fidelity images, while also offering unprecedented semantic editing. However, these fascinating abilities have been demonstrated only on a limited set of datasets, which are usually structurally aligned and well curated. In this paper, we show how StyleGAN can be adapted to work on raw uncurated images collected from the Internet. Such image collections impose two main challenges to StyleGAN: they contain many outlier images, and are characterized by a multi-modal distribution. Training StyleGAN on such raw image collections results in degraded image synthesis quality. To meet these challenges, we proposed a StyleGAN-based self-distillation approach, which consists of two main components: (i) A generative-based self-filtering of the dataset to eliminate outlier images, in order to generate an adequate training set, and (ii) Perceptual clustering of the generated images to detect the inherent data modalities, which are then employed to improve StyleGAN's "truncation trick" in the image synthesis process. The presented technique enables the generation of high-quality images, while minimizing the loss in diversity of the data. Through qualitative and quantitative evaluation, we demonstrate the power of our approach to new challenging and diverse domains collected from the Internet. New datasets and pre-trained models are available at https://self-distilled-stylegan.github.io/ .