Neural Photo Editing with Introspective Adversarial Networks

TL;DR

This paper introduces the Neural Photo Editor, utilizing Introspective Adversarial Networks that combine VAE and GAN techniques to enable high-quality, semantically coherent edits to images, validated on multiple datasets.

Contribution

The paper proposes a novel hybrid model, Introspective Adversarial Network, combining VAE and GAN, with a new regularization method and a specialized convolutional block for improved image reconstruction.

Findings

High visual fidelity in generated samples and reconstructions

Effective semantic image editing demonstrated on CelebA, SVHN, CIFAR-100

Improved generalization performance with Orthogonal Regularization

Abstract

The increasingly photorealistic sample quality of generative image models suggests their feasibility in applications beyond image generation. We present the Neural Photo Editor, an interface that leverages the power of generative neural networks to make large, semantically coherent changes to existing images. To tackle the challenge of achieving accurate reconstructions without loss of feature quality, we introduce the Introspective Adversarial Network, a novel hybridization of the VAE and GAN. Our model efficiently captures long-range dependencies through use of a computational block based on weight-shared dilated convolutions, and improves generalization performance with Orthogonal Regularization, a novel weight regularization method. We validate our contributions on CelebA, SVHN, and CIFAR-100, and produce samples and reconstructions with high visual fidelity.