Blind Inversion using Latent Diffusion Priors
Weimin Bai, Siyi Chen, Wenzheng Chen, He Sun
TL;DR
This paper introduces LatentDEM, a novel method for blind inverse problems using latent diffusion priors within an EM framework, enabling effective 2D and 3D inverse tasks without known forward operators.
Contribution
LatentDEM is the first approach to combine latent diffusion models with EM for blind inverse problems, supporting both linear and non-linear cases.
Findings
Superior performance on 2D blind deblurring
Effective in 3D sparse-view reconstruction
Outperforms prior methods in accuracy and efficiency
Abstract
Diffusion models have emerged as powerful tools for solving inverse problems due to their exceptional ability to model complex prior distributions. However, existing methods predominantly assume known forward operators (i.e., non-blind), limiting their applicability in practical settings where acquiring such operators is costly. Additionally, many current approaches rely on pixel-space diffusion models, leaving the potential of more powerful latent diffusion models (LDMs) underexplored. In this paper, we introduce LatentDEM, an innovative technique that addresses more challenging blind inverse problems using latent diffusion priors. At the core of our method is solving blind inverse problems within an iterative Expectation-Maximization (EM) framework: (1) the E-step recovers clean images from corrupted observations using LDM priors and a known forward model, and (2) the M-step estimates…
Peer Reviews
Decision·Submitted to ICLR 2025
This problem holds broad interest across the machine learning and signal processing communities. The paper is generally well-structured and somewhat clear.
Several technical ambiguities require clarification. These include the choice of likelihood formulation, the role of “gluing” regularization versus annealing consistency, and the performance discrepancies across datasets. More experimental validation, particularly isolating the effects of different regularization techniques, would strengthen the claims. Additionally, a detailed comparison of hyperparameters with competing methods would improve transparency and reproducibility. For further inform
- As far as I know, this is the first work that uses a pretrained LDM for blind restoration. - The proposed method appears to have empirical advantages over competitors.
- The contribution seems incremental in terms of the techniques being used and the lack of theoretical motivation and analysis of the proposed method. - The presentation is not good in terms of explaining why the authors refer to the proposed method as an expectation maximization (EM) technique.
The paper is well-organized and clearly communicates its results. The authors provide extensive experimental results, comparing their method to prominent existing methods for blind inverse problems, such as BlindDPS and FastEM. They provide an ablation study examining the influence of gradient estimation skipping and annealing consistency on performance.
1. A primary concern is the paper’s original contribution. The methodology appears to combine elements previously explored in the literature. For instance, latent diffusion models for inverse problems have been investigated in [1], while Expectation-Maximization approaches for blind inverse problems have been studied in [2]. Simply substituting a latent diffusion model as the prior does not, in itself, seem to constitute a novel contribution. Explicitly listing the contributions could improve cl
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsSpeech Recognition and Synthesis · Seismic Imaging and Inversion Techniques · Advanced Data Compression Techniques
MethodsDiffusion