Differentiable Gaussianization Layers for Inverse Problems Regularized by Deep Generative Models

TL;DR

This paper introduces differentiable Gaussianization layers to reparameterize and constrain latent tensors in deep generative models, improving the fidelity of solutions in inverse problems like MRI, deblurring, and tomography.

Contribution

It proposes novel data-dependent layers that reparameterize and Gaussianize latent tensors, enhancing the quality of inverse problem solutions with deep generative models.

Findings

Achieves state-of-the-art accuracy in MRI, deblurring, and tomography tasks.

Effectively maintains in-distribution solutions during inversion.

Improves fidelity and consistency of reconstructed images.

Abstract

Deep generative models such as GANs, normalizing flows, and diffusion models are powerful regularizers for inverse problems. They exhibit great potential for helping reduce ill-posedness and attain high-quality results. However, the latent tensors of such deep generative models can fall out of the desired high-dimensional standard Gaussian distribution during inversion, particularly in the presence of data noise and inaccurate forward models, leading to low-fidelity solutions. To address this issue, we propose to reparameterize and Gaussianize the latent tensors using novel differentiable data-dependent layers wherein custom operators are defined by solving optimization problems. These proposed layers constrain inverse problems to obtain high-fidelity in-distribution solutions. We validate our technique on three inversion tasks: compressive-sensing MRI, image deblurring, and eikonal tomography (a nonlinear PDE-constrained inverse problem) using two representative deep generative models: StyleGAN2 and Glow. Our approach achieves state-of-the-art performance in terms of accuracy and consistency.