DLGAN: Disentangling Label-Specific Fine-Grained Features for Image Manipulation

TL;DR

DLGAN introduces a novel framework that disentangles label-specific features for controllable and hybrid image manipulation, enabling smooth interpolation and domain transfer without continuous labels.

Contribution

It is the first model to perform hybrid manipulation using discrete multi-labels and reference images, allowing domain interpolation without continuous supervision.

Findings

Effective disentanglement of label-specific features.

Smooth image interpolation between different domains.

Controlled manipulation of attributes like gender and age.

Abstract

Recent studies have shown how disentangling images into content and feature spaces can provide controllable image translation/ manipulation. In this paper, we propose a framework to enable utilizing discrete multi-labels to control which features to be disentangled, i.e., disentangling label-specific fine-grained features for image manipulation (dubbed DLGAN). By mapping the discrete label-specific attribute features into a continuous prior distribution, we leverage the advantages of both discrete labels and reference images to achieve image manipulation in a hybrid fashion. For example, given a face image dataset (e.g., CelebA) with multiple discrete fine-grained labels, we can learn to smoothly interpolate a face image between black hair and blond hair through reference images while immediately controlling the gender and age through discrete input labels. To the best of our knowledge, this is the first work that realizes such a hybrid manipulation within a single model. More importantly, it is the first work to achieve image interpolation between two different domains without requiring continuous labels as the supervision. Qualitative and quantitative experiments demonstrate the effectiveness of the proposed method.