Spatial PixelCNN: Generating Images from Patches

TL;DR

Spatial PixelCNN is a novel model that generates high-quality images from patches by conditioning on pixel coordinates and global features, enabling arbitrary resolution upscaling.

Contribution

It introduces a patch-based autoregressive model conditioned on coordinates and features, allowing for flexible image generation and upscaling.

Findings

Achieves high-quality image generation from patches.

Capable of upscaling images to arbitrary resolutions.

Performs comparably to PixelCNN++ on MNIST.

Abstract

In this paper we propose Spatial PixelCNN, a conditional autoregressive model that generates images from small patches. By conditioning on a grid of pixel coordinates and global features extracted from a Variational Autoencoder (VAE), we are able to train on patches of images, and reproduce the full-sized image. We show that it not only allows for generating high quality samples at the same resolution as the underlying dataset, but is also capable of upscaling images to arbitrary resolutions (tested at resolutions up to $50\times$) on the MNIST dataset. Compared to a PixelCNN++ baseline, Spatial PixelCNN quantitatively and qualitatively achieves similar performance on the MNIST dataset.