Auxiliary Guided Autoregressive Variational Autoencoders

TL;DR

This paper introduces an auxiliary guided training method for autoregressive variational autoencoders, enabling the use of powerful decoders while effectively capturing global image structure in latent variables, leading to improved performance.

Contribution

It proposes a novel auxiliary loss-based training procedure that balances information between latent variables and autoregressive decoders, enhancing model capacity and sample quality.

Findings

Achieves state-of-the-art results among latent variable models.

Enables use of arbitrarily powerful autoregressive decoders.

Produces qualitatively convincing image samples.

Abstract

Generative modeling of high-dimensional data is a key problem in machine learning. Successful approaches include latent variable models and autoregressive models. The complementary strengths of these approaches, to model global and local image statistics respectively, suggest hybrid models that encode global image structure into latent variables while autoregressively modeling low level detail. Previous approaches to such hybrid models restrict the capacity of the autoregressive decoder to prevent degenerate models that ignore the latent variables and only rely on autoregressive modeling. Our contribution is a training procedure relying on an auxiliary loss function that controls which information is captured by the latent variables and what is left to the autoregressive decoder. Our approach can leverage arbitrarily powerful autoregressive decoders, achieves state-of-the art quantitative performance among models with latent variables, and generates qualitatively convincing samples.