Integrating Intermediate Layer Optimization and Projected Gradient Descent for Solving Inverse Problems with Diffusion Models

TL;DR

This paper introduces DMILO and DMILO-PGD, two novel methods that improve diffusion model-based inverse problem solving by reducing computational demands and enhancing convergence, validated through extensive experiments.

Contribution

The paper proposes intermediate layer optimization and sparse deviations to expand diffusion model capabilities and integrates projected gradient descent to improve convergence in inverse problem solutions.

Findings

Significant performance improvements over state-of-the-art methods.

Reduced memory usage compared to previous diffusion model approaches.

Enhanced ability to explore signals outside the diffusion model's original range.

Abstract

Inverse problems (IPs) involve reconstructing signals from noisy observations. Recently, diffusion models (DMs) have emerged as a powerful framework for solving IPs, achieving remarkable reconstruction performance. However, existing DM-based methods frequently encounter issues such as heavy computational demands and suboptimal convergence. In this work, building upon the idea of the recent work DMPlug, we propose two novel methods, DMILO and DMILO-PGD, to address these challenges. Our first method, DMILO, employs intermediate layer optimization (ILO) to alleviate the memory burden inherent in DMPlug. Additionally, by introducing sparse deviations, we expand the range of DMs, enabling the exploration of underlying signals that may lie outside the range of the diffusion model. We further propose DMILO-PGD, which integrates ILO with projected gradient descent (PGD), thereby reducing the risk of suboptimal convergence. We provide an intuitive theoretical analysis of our approaches under appropriate conditions and validate their superiority through extensive experiments on diverse image datasets, encompassing both linear and nonlinear IPs. Our results demonstrate significant performance gains over state-of-the-art methods, highlighting the effectiveness of DMILO and DMILO-PGD in addressing common challenges in DM-based IP solvers.