Enhancing Diffusion Posterior Sampling for Inverse Problems by Integrating Crafted Measurements
Shijie Zhou, Huaisheng Zhu, Rohan Sharma, Jiayi Chen, Ruiyi Zhang, Kaiyi Ji, Changyou Chen
TL;DR
This paper introduces DPS-CM, a novel diffusion posterior sampling method that integrates crafted measurements to improve inverse problem solving, especially under noisy conditions, by reducing early-stage high-frequency errors.
Contribution
The paper proposes a new diffusion posterior sampling technique that uses crafted measurements to better align with the diffusion prior, enhancing performance on noisy inverse problems.
Findings
Significant improvement in solving inverse problems like deblurring and super-resolution.
Effective mitigation of early-stage high-frequency information errors.
Enhanced robustness to noisy measurements in inverse problem tasks.
Abstract
Diffusion models have emerged as a powerful foundation model for visual generations. With an appropriate sampling process, it can effectively serve as a generative prior for solving general inverse problems. Current posterior sampling-based methods take the measurement (i.e., degraded image sample) into the posterior sampling to infer the distribution of the target data (i.e., clean image sample). However, in this manner, we show that high-frequency information can be prematurely introduced during the early stages, which could induce larger posterior estimate errors during restoration sampling. To address this observation, we first reveal that forming the log-posterior gradient with the noisy measurement ( i.e., noisy measurement from a diffusion forward process) instead of the clean one can benefit the early posterior sampling. Consequently, we propose a novel diffusion posterior…
Peer Reviews
Decision·ICLR 2025 Conference Withdrawn Submission
The proposed algorithm is intuitively appealing, as the additional gradient that guides toward the smoothed measurement $\hat{y}_0$ aids in achieving better reconstruction of low-frequency components. The algorithm is evaluated across various experiments on two datasets using several performance metrics. In all presented experiments, the proposed method demonstrates improvements over [1].
Section 3 raises some questions that, I hope, the authors can help clarify. First, could the authors clarify the difference between $y_t$ in line 291 (noisy measurements) and $\text{y}_t$ as defined in line 295 (crafted measurements)? As I understand it, noisy measurements are sampled from the forward trajectory of the $\\{y_t\\}$ diffusion process, while the crafted measurements are generated during the reverse-time trajectory. Does the direction of the diffusion impact the definition of these
1, The manuscript is well-written in most of its part and easy to follow in its motivation behind the designed crafted sampling trick. The numerical validation of this motivation to design $\nabla \log p(x_t|y_t)$ is interesting and insightful. 2, This sampling trick is easy to implement without too many changes of original DPS method. 3, The numerical validation shows the effectiveness of this proposal.
1, The proposed crafted measurement $y_t$ in lines 3–6 of Alg1. is based on an underlying assumption that the measurement sample lies within the domain of the deep score network $s_\theta$. However, this assumption may not hold for various inverse problems, such as CT/MR imaging and phase retrieval, where the measurement distribution often differs from that of the training images $x_0$. This discrepancy may fundamentally limit the practical contribution of the DPS-CM method, as it is unclear how
- The intuition and motivations are clearly explained.
In my opinion this paper is technically flawed, incoherent in some places and contains no review of the relevant and close literature. - First, the presentation of the methodology starts with "We propose DPS-CM, which perform diffusion posterior sampling from $p(x_t | y_t)$ with Crafted Measurement $y_t$ belonging to another diffusion reverse trajectory $( y_t )^T _{t=0}$ instead of the vanilla input $y$". In this statement it is not clear at all what "posterior sampling" from $p(x_t | y_t )$
1. The core contribution would be the combination of DDPM reverse trajectory and corrupted measurements as conditions of the score function. 2. The paper is well-written and easy to follow. 3. A comprehensive study of posterior sampling errors is performed.
1. The contribution seems incremental. 2. Intuition for designing the crafted measurements should be provided. A visualization of the measurements during the sample trajectory is also helpful for the readers to follow. 3. The method relies heavily on the crafted measurement. The result may be unstable due to different crafted measurements sampled from the reverse DDPM process.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsNumerical methods in inverse problems
MethodsDiffusion