Conditional Image Generation with PixelCNN Decoders

TL;DR

This paper introduces a conditional PixelCNN model capable of generating diverse and realistic images based on various conditioning inputs, improving image generation quality and flexibility.

Contribution

The paper presents a novel conditional PixelCNN architecture that can incorporate different conditioning vectors, serving as a powerful decoder and achieving state-of-the-art likelihoods with reduced computation.

Findings

Generated diverse images conditioned on class labels and embeddings

Matched state-of-the-art likelihood performance with lower computational cost

Produced realistic portraits with varied expressions and lighting

Abstract

This work explores conditional image generation with a new image density model based on the PixelCNN architecture. The model can be conditioned on any vector, including descriptive labels or tags, or latent embeddings created by other networks. When conditioned on class labels from the ImageNet database, the model is able to generate diverse, realistic scenes representing distinct animals, objects, landscapes and structures. When conditioned on an embedding produced by a convolutional network given a single image of an unseen face, it generates a variety of new portraits of the same person with different facial expressions, poses and lighting conditions. We also show that conditional PixelCNN can serve as a powerful decoder in an image autoencoder. Additionally, the gated convolutional layers in the proposed model improve the log-likelihood of PixelCNN to match the state-of-the-art performance of PixelRNN on ImageNet, with greatly reduced computational cost.