Deep Data Consistency: a Fast and Robust Diffusion Model-based Solver for Inverse Problems

TL;DR

This paper introduces Deep Data Consistency (DDC), a diffusion model-based solver that achieves fast, robust, and high-quality solutions for inverse problems with minimal inference steps, outperforming existing methods.

Contribution

The paper proposes DDC, a novel deep learning approach that enhances diffusion models for inverse problems, balancing data consistency and realness efficiently with only 5 inference steps.

Findings

Outperforms state-of-the-art methods in linear and non-linear tasks.

Generates high-quality solutions in 0.77 seconds on average.

Demonstrates robustness across datasets and noise levels.

Abstract

Diffusion models have become a successful approach for solving various image inverse problems by providing a powerful diffusion prior. Many studies tried to combine the measurement into diffusion by score function replacement, matrix decomposition, or optimization algorithms, but it is hard to balance the data consistency and realness. The slow sampling speed is also a main obstacle to its wide application. To address the challenges, we propose Deep Data Consistency (DDC) to update the data consistency step with a deep learning model when solving inverse problems with diffusion models. By analyzing existing methods, the variational bound training objective is used to maximize the conditional posterior and reduce its impact on the diffusion process. In comparison with state-of-the-art methods in linear and non-linear tasks, DDC demonstrates its outstanding performance of both similarity and realness metrics in generating high-quality solutions with only 5 inference steps in 0.77 seconds on average. In addition, the robustness of DDC is well illustrated in the experiments across datasets, with large noise and the capacity to solve multiple tasks in only one pre-trained model.