Score-Guided Intermediate Layer Optimization: Fast Langevin Mixing for Inverse Problems

TL;DR

This paper introduces SGILO, a novel method for inverse problems that leverages score-based models in the latent space of generative networks, achieving faster mixing and improved results over previous methods.

Contribution

It extends Langevin Algorithm analysis to posterior sampling in deep generative models and proposes a new regularization approach using intermediate layer priors.

Findings

Significant improvement over state-of-the-art in low measurement regimes

Proven fast mixing and characterized stationary distribution of Langevin Algorithm

Effective posterior sampling in the latent space of StyleGAN-2

Abstract

We prove fast mixing and characterize the stationary distribution of the Langevin Algorithm for inverting random weighted DNN generators. This result extends the work of Hand and Voroninski from efficient inversion to efficient posterior sampling. In practice, to allow for increased expressivity, we propose to do posterior sampling in the latent space of a pre-trained generative model. To achieve that, we train a score-based model in the latent space of a StyleGAN-2 and we use it to solve inverse problems. Our framework, Score-Guided Intermediate Layer Optimization (SGILO), extends prior work by replacing the sparsity regularization with a generative prior in the intermediate layer. Experimentally, we obtain significant improvements over the previous state-of-the-art, especially in the low measurement regime.