Labels4Free: Unsupervised Segmentation using StyleGAN

TL;DR

This paper introduces an unsupervised segmentation method for StyleGAN-generated images by augmenting the generator with a segmentation branch, enabling effective foreground-background separation without labeled data.

Contribution

It presents a novel unsupervised segmentation framework that splits StyleGAN2 into foreground and background networks with a segmentation branch, improving over existing methods.

Findings

Achieves comparable results to supervised segmentation networks.

Outperforms existing unsupervised segmentation approaches.

Provides high-quality soft segmentation masks without labeled data.

Abstract

We propose an unsupervised segmentation framework for StyleGAN generated objects. We build on two main observations. First, the features generated by StyleGAN hold valuable information that can be utilized towards training segmentation networks. Second, the foreground and background can often be treated to be largely independent and be composited in different ways. For our solution, we propose to augment the StyleGAN2 generator architecture with a segmentation branch and to split the generator into a foreground and background network. This enables us to generate soft segmentation masks for the foreground object in an unsupervised fashion. On multiple object classes, we report comparable results against state-of-the-art supervised segmentation networks, while against the best unsupervised segmentation approach we demonstrate a clear improvement, both in qualitative and quantitative metrics.