Representation Decomposition for Image Manipulation and Beyond

TL;DR

This paper introduces dec-GAN, a novel method for decomposing existing latent representations into content and attribute features, enabling better image manipulation without retraining the generative model.

Contribution

The proposed dec-GAN can decompose pre-existing latent representations into interpretable features, unlike prior methods that require disjoint attribute codes and retraining.

Findings

Dec-GAN effectively decomposes latent representations on multiple datasets.

Dec-GAN outperforms recent disentanglement models in experiments.

The method demonstrates robustness and applicability to existing trained models.

Abstract

Representation disentanglement aims at learning interpretable features, so that the output can be recovered or manipulated accordingly. While existing works like infoGAN and AC-GAN exist, they choose to derive disjoint attribute code for feature disentanglement, which is not applicable for existing/trained generative models. In this paper, we propose a decomposition-GAN (dec-GAN), which is able to achieve the decomposition of an existing latent representation into content and attribute features. Guided by the classifier pre-trained on the attributes of interest, our dec-GAN decomposes the attributes of interest from the latent representation, while data recovery and feature consistency objectives enforce the learning of our proposed method. Our experiments on multiple image datasets confirm the effectiveness and robustness of our dec-GAN over recent representation disentanglement models.