ContraFeat: Contrasting Deep Features for Semantic Discovery

TL;DR

This paper introduces ContraFeat, an automated method for semantic discovery in StyleGAN that leverages deep feature contrasts and attention mechanisms to outperform manual layer selection approaches.

Contribution

The paper proposes a novel automated semantic discovery model with attention and deep feature contrast losses, achieving state-of-the-art results without manual layer selection.

Findings

State-of-the-art semantic discovery performance.

Effective manipulation of real-world images.

Quantitative metrics for semantic discovery evaluation.

Abstract

StyleGAN has shown strong potential for disentangled semantic control, thanks to its special design of multi-layer intermediate latent variables. However, existing semantic discovery methods on StyleGAN rely on manual selection of modified latent layers to obtain satisfactory manipulation results, which is tedious and demanding. In this paper, we propose a model that automates this process and achieves state-of-the-art semantic discovery performance. The model consists of an attention-equipped navigator module and losses contrasting deep-feature changes. We propose two model variants, with one contrasting samples in a binary manner, and another one contrasting samples with learned prototype variation patterns. The proposed losses are defined with pretrained deep features, based on our assumption that the features can implicitly reveal the desired semantic structure including consistency and orthogonality. Additionally, we design two metrics to quantitatively evaluate the performance of semantic discovery methods on FFHQ dataset, and also show that disentangled representations can be derived via a simple training process. Experimentally, our models can obtain state-of-the-art semantic discovery results without relying on latent layer-wise manual selection, and these discovered semantics can be used to manipulate real-world images.