PixelPyramids: Exact Inference Models from Lossless Image Pyramids

TL;DR

PixelPyramids introduce a novel block-autoregressive model using lossless image pyramids, achieving state-of-the-art density estimation especially for high-resolution images with improved efficiency.

Contribution

The paper presents PixelPyramids, a new scalable autoregressive model that reduces dependency complexity using lossless pyramids, enabling high-resolution image density estimation.

Findings

State-of-the-art density estimates on high-resolution images.

Improved sampling speed over traditional autoregressive models.

Enhanced efficiency with sparser dependency structures.

Abstract

Autoregressive models are a class of exact inference approaches with highly flexible functional forms, yielding state-of-the-art density estimates for natural images. Yet, the sequential ordering on the dimensions makes these models computationally expensive and limits their applicability to low-resolution imagery. In this work, we propose Pixel-Pyramids, a block-autoregressive approach employing a lossless pyramid decomposition with scale-specific representations to encode the joint distribution of image pixels. Crucially, it affords a sparser dependency structure compared to fully autoregressive approaches. Our PixelPyramids yield state-of-the-art results for density estimation on various image datasets, especially for high-resolution data. For CelebA-HQ 1024 x 1024, we observe that the density estimates (in terms of bits/dim) are improved to ~44% of the baseline despite sampling speeds superior even to easily parallelizable flow-based models.