Supervised Attribute Information Removal and Reconstruction for Image Manipulation

TL;DR

This paper introduces AIRR, a novel network that effectively removes attribute information from images to improve attribute manipulation accuracy and image quality, outperforming prior methods across multiple datasets.

Contribution

The paper proposes the AIRR network that removes attribute information before injecting desired attributes, addressing information hiding issues in disentangled representations.

Findings

Improves attribute manipulation accuracy by 10% on average.

Enhances top-k retrieval rate by 10%.

User study shows 76% preference for AIRR-manipulated images.

Abstract

The goal of attribute manipulation is to control specified attribute(s) in given images. Prior work approaches this problem by learning disentangled representations for each attribute that enables it to manipulate the encoded source attributes to the target attributes. However, encoded attributes are often correlated with relevant image content. Thus, the source attribute information can often be hidden in the disentangled features, leading to unwanted image editing effects. In this paper, we propose an Attribute Information Removal and Reconstruction (AIRR) network that prevents such information hiding by learning how to remove the attribute information entirely, creating attribute excluded features, and then learns to directly inject the desired attributes in a reconstructed image. We evaluate our approach on four diverse datasets with a variety of attributes including DeepFashion Synthesis, DeepFashion Fine-grained Attribute, CelebA and CelebA-HQ, where our model improves attribute manipulation accuracy and top-k retrieval rate by 10% on average over prior work. A user study also reports that AIRR manipulated images are preferred over prior work in up to 76% of cases.