Inverse problem regularization with hierarchical variational autoencoders

TL;DR

This paper introduces a hierarchical variational autoencoder-based regularization method for ill-posed inverse problems, combining advantages of denoiser-based and generative model approaches, with proven convergence and applicability to natural images.

Contribution

The paper develops a PnP-HVAE method that leverages hierarchical VAEs for inverse problem regularization, offering convergence guarantees and broad applicability to natural images.

Findings

Competitive with state-of-the-art denoiser-based PnP methods

Outperforms some existing generative model approaches

Applicable to images of any size

Abstract

In this paper, we propose to regularize ill-posed inverse problems using a deep hierarchical variational autoencoder (HVAE) as an image prior. The proposed method synthesizes the advantages of i) denoiser-based Plug \& Play approaches and ii) generative model based approaches to inverse problems. First, we exploit VAE properties to design an efficient algorithm that benefits from convergence guarantees of Plug-and-Play (PnP) methods. Second, our approach is not restricted to specialized datasets and the proposed PnP-HVAE model is able to solve image restoration problems on natural images of any size. Our experiments show that the proposed PnP-HVAE method is competitive with both SOTA denoiser-based PnP approaches, and other SOTA restoration methods based on generative models.