Diverse super-resolution with pretrained deep hiererarchical VAEs

TL;DR

This paper introduces a method for diverse image super-resolution using a pretrained hierarchical VAE as a prior, combining efficiency and quality for face super-resolution tasks.

Contribution

It proposes a novel approach that leverages a pretrained HVAE and a lightweight encoder to generate diverse high-resolution images efficiently.

Findings

Balances computational efficiency and sample quality

Effective for face super-resolution tasks

Utilizes a probabilistic framework for diversity

Abstract

We investigate the problem of producing diverse solutions to an image super-resolution problem. From a probabilistic perspective, this can be done by sampling from the posterior distribution of an inverse problem, which requires the definition of a prior distribution on the high-resolution images. In this work, we propose to use a pretrained hierarchical variational autoencoder (HVAE) as a prior. We train a lightweight stochastic encoder to encode low-resolution images in the latent space of a pretrained HVAE. At inference, we combine the low-resolution encoder and the pretrained generative model to super-resolve an image. We demonstrate on the task of face super-resolution that our method provides an advantageous trade-off between the computational efficiency of conditional normalizing flows techniques and the sample quality of diffusion based methods.