Posterior sampling via Langevin dynamics based on generative priors
Vishal Purohit, Matthew Repasky, Jianfeng Lu, Qiang Qiu, Yao Xie,, Xiuyuan Cheng
TL;DR
This paper introduces an efficient method for posterior sampling in high-dimensional generative models by simulating Langevin dynamics in noise space, reducing computational costs while maintaining sample diversity and fidelity.
Contribution
The authors propose a novel noise-space Langevin dynamics approach that enables seamless posterior exploration without re-running the full generative process, backed by theoretical guarantees.
Findings
Achieves high-fidelity, diverse samples with fewer function evaluations.
Outperforms existing diffusion-based methods in efficiency and quality.
Validated on image restoration tasks with real datasets.
Abstract
Posterior sampling in high-dimensional spaces using generative models holds significant promise for various applications, including but not limited to inverse problems and guided generation tasks. Despite many recent developments, generating diverse posterior samples remains a challenge, as existing methods require restarting the entire generative process for each new sample, making the procedure computationally expensive. In this work, we propose efficient posterior sampling by simulating Langevin dynamics in the noise space of a pre-trained generative model. By exploiting the mapping between the noise and data spaces which can be provided by distilled flows or consistency models, our method enables seamless exploration of the posterior without the need to re-run the full sampling chain, drastically reducing computational overhead. Theoretically, we prove a guarantee for the proposed…
Peer Reviews
Decision·ICLR 2025 Conference Withdrawn Submission
* The paper is well written and easy to follow. * Projecting the posterior sampling to a noise space is a good idea, as the posterior distribution in the noise space is close to a single modal Gaussian distribution, that is more friendly to MCMC. * Theoretical analysis has been provided for the proposed method.
* My main concern is that the paper fails to put itself to the current position in the literature. This type of projecting MCMC sampling to a more MCMC friendly noise space has been well established in [1], and has been later on adapted to generative modeling regime in e.g. [2, 3], with the deterministic mapping being a VAE or a flow-based model. The contribution of this paper, positioned in these literation, is that it adapted the sampling to a posterior distribution, and leverages a CM with fi
This paper has proposed an efficient posterior sampling method that could avoid running the full sampling chain.
1. The proposed method of posterior sampling using Langevin dynamics has been extensively studied and applied within the context of energy-based models. As a result, this approach lacks sufficient novelty for this work, given the established body of research already dedicated to similar methodologies. 2. The proposed method of sampling by posterior sampling by Langevin dynamics has been early explored in many EBM works such as [1] - [4] for multiple kinds of tasks, such as image generation, tra
- Well-motivated problem, the authors did a good literature review that lists relevant works. - The derviation of the framework is based on establishing theories of SMC/denoising diffusion models. A theoretical analysis is always welcomed. - The method is relatively straight-forward and easy to implement, and work on both linear and non-linear inverse problem settings.
- Huge doubt about practical performance (inference time): while the authors report low NFE, the total runtime of the sampling framework is not reported. The backpropagation through the whole pretrained consistency model $\phi$ for each steps are costly in both memory and computational time. I think it will be fairer to compare total wall-time with other baselines instead of just listing the NFEs as stated in the paper - Unclear about the advantages of the proposed methods vs. baselines used in
- The paper is well-written and easy to follow. The idea is simple but effective. - The leveraging of consistency models significantly reduces the sampling costs, which is better than the previous data-prediction diffusion models. - The experiments are solid.
This paper doesn't have major weakness, though there are two minor aspects: - The method is slightly lack of novelty since the combination of consistency models and Lagevin dynamics is a direct generation of previous method such as DPS. - The proposed method can only tackle with simple likelihood functions such as inpainting / deblurring / super resolution. It is unclear whether the proposed method can do complicated posterior inference in traditional Bayesian inference settings.
**Originality** : to the best of my knowledge, this paper is the first to perform posterior sampling in the latent space of consistency models. **Quality** : the proposed method is theoretically sound, and claims for diverse posterior sampling is supported with experiments on ImageNet 64x64 and LSUN bedroom 256x256. **Clarity** : the proposed method and main results are clearly presented. I had no problem following the exposition. **Significance** : while current diffusion-based image restora
**Limited Originality** : running MCMC in the latent space of one-step pushforward generative models is not new. Similar ideas are already explored in works such as [1,2] (none of which are cited in this paper). I also feel the paper is limited in its technical novelty, as it does not provide any insight into efficient posterior sampling in the latent space of more challenging multi-step priors such as diffusion models or hierarchical variational autoencoders. **Overstatements** : I feel that t
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsMachine Learning and Algorithms · Blind Source Separation Techniques · Bayesian Methods and Mixture Models