Pixel Recurrent Neural Networks

TL;DR

This paper introduces Pixel RNNs, a deep neural network that sequentially models image pixels to capture complex dependencies, achieving state-of-the-art likelihood scores and producing high-quality, coherent image samples.

Contribution

It presents a novel 2D recurrent neural network architecture with residual connections, improving image modeling and setting new benchmarks on ImageNet.

Findings

Achieves significantly better log-likelihood scores than previous models.

Generates crisp, varied, and globally coherent images.

Provides new benchmarks on the ImageNet dataset.

Abstract

Modeling the distribution of natural images is a landmark problem in unsupervised learning. This task requires an image model that is at once expressive, tractable and scalable. We present a deep neural network that sequentially predicts the pixels in an image along the two spatial dimensions. Our method models the discrete probability of the raw pixel values and encodes the complete set of dependencies in the image. Architectural novelties include fast two-dimensional recurrent layers and an effective use of residual connections in deep recurrent networks. We achieve log-likelihood scores on natural images that are considerably better than the previous state of the art. Our main results also provide benchmarks on the diverse ImageNet dataset. Samples generated from the model appear crisp, varied and globally coherent.