Unleashing the Denoising Capability of Diffusion Prior for Solving Inverse Problems

TL;DR

This paper introduces ProjDiff, a novel optimization algorithm that leverages diffusion models' denoising capabilities for solving inverse problems more effectively, demonstrating superior performance in image restoration and source separation tasks.

Contribution

It proposes a new framework that combines diffusion priors with constrained optimization, utilizing gradient truncation to enhance inverse problem solving.

Findings

ProjDiff outperforms existing methods in image restoration tasks.

The approach effectively utilizes diffusion models' denoising capabilities.

Experimental results show improved accuracy in various inverse problems.

Abstract

The recent emergence of diffusion models has significantly advanced the precision of learnable priors, presenting innovative avenues for addressing inverse problems. Since inverse problems inherently entail maximum a posteriori estimation, previous works have endeavored to integrate diffusion priors into the optimization frameworks. However, prevailing optimization-based inverse algorithms primarily exploit the prior information within the diffusion models while neglecting their denoising capability. To bridge this gap, this work leverages the diffusion process to reframe noisy inverse problems as a two-variable constrained optimization task by introducing an auxiliary optimization variable. By employing gradient truncation, the projection gradient descent method is efficiently utilized to solve the corresponding optimization problem. The proposed algorithm, termed ProjDiff, effectively harnesses the prior information and the denoising capability of a pre-trained diffusion model within the optimization framework. Extensive experiments on the image restoration tasks and source separation and partial generation tasks demonstrate that ProjDiff exhibits superior performance across various linear and nonlinear inverse problems, highlighting its potential for practical applications. Code is available at https://github.com/weigerzan/ProjDiff/.