pcaGAN: Improving Posterior-Sampling cGANs via Principal Component Regularization

TL;DR

pcaGAN introduces a novel regularization technique for cGANs that enhances posterior sampling accuracy in inverse imaging problems, outperforming existing models in tasks like denoising and MRI recovery.

Contribution

It proposes a new principal component regularization method for cGANs to improve posterior mean and covariance estimation in inverse problems.

Findings

Outperforms contemporary cGANs and diffusion models in various imaging tasks.

Accurately estimates posterior mean and principal components.

Demonstrates effectiveness in denoising, inpainting, and MRI recovery.

Abstract

In ill-posed imaging inverse problems, there can exist many hypotheses that fit both the observed measurements and prior knowledge of the true image. Rather than returning just one hypothesis of that image, posterior samplers aim to explore the full solution space by generating many probable hypotheses, which can later be used to quantify uncertainty or construct recoveries that appropriately navigate the perception/distortion trade-off. In this work, we propose a fast and accurate posterior-sampling conditional generative adversarial network (cGAN) that, through a novel form of regularization, aims for correctness in the posterior mean as well as the trace and K principal components of the posterior covariance matrix. Numerical experiments demonstrate that our method outperforms contemporary cGANs and diffusion models in imaging inverse problems like denoising, large-scale inpainting, and accelerated MRI recovery. The code for our model can be found here: https://github.com/matt-bendel/pcaGAN.