Conditional Variational Diffusion Models
Gabriel della Maggiora, Luis Alberto Croquevielle, Nikita Deshpande, Harry Horsley, Thomas Heinis, Artur Yakimovich
TL;DR
This paper introduces a method to learn the variance schedule of diffusion models during training, enhancing their adaptability and performance in inverse problems like microscopy and imaging.
Contribution
It presents a novel approach for jointly learning the diffusion schedule with model training, eliminating the need for costly fine-tuning for specific applications.
Findings
Achieves high-quality solutions comparable or superior to fine-tuned models.
Demonstrates flexibility across different inverse problems.
Shows stable training process for schedule learning.
Abstract
Inverse problems aim to determine parameters from observations, a crucial task in engineering and science. Lately, generative models, especially diffusion models, have gained popularity in this area for their ability to produce realistic solutions and their good mathematical properties. Despite their success, an important drawback of diffusion models is their sensitivity to the choice of variance schedule, which controls the dynamics of the diffusion process. Fine-tuning this schedule for specific applications is crucial but time-costly and does not guarantee an optimal result. We propose a novel approach for learning the schedule as part of the training process. Our method supports probabilistic conditioning on data, provides high-quality solutions, and is flexible, proving able to adapt to different applications with minimum overhead. This approach is tested in two unrelated inverse…
Peer Reviews
Decision·ICLR 2024 poster
The paper is well written and quite easy to understand, which is great. The numerical experiments/applications are interesting and show that the proposed method provides comparable or superior performance to existing methods that fine-tune the schedule.
I think that the contribution of this paper is quite marginal and is not suited for ICLR. The main difference with the methodology developed in [1] is the replacement of the unstable SNR term. While this yields good results in practice, I am not sure if this is enough novelty for a conference like ICLR.
This paper's primary strength lies in its practical extension of variational diffusion models (VDM) to the conditioning case, including several technical improvements such as the incorporation of a regularization term on the signal-to-noise ratio. In addition, by adopting the VDM framework, the paper eliminates the need of prior works that fine tunes the variance schedule. Another strength of the paper is on the experiment study which includes two practical downstream benchmarks assessed wit
The biggest weakness is the paper's technical novelty. The proposed approach seems like a straightforward conditional extension to the of variational diffusion models. In addition, the paradigm of turning an unconditional model to a conditional version has been largely explored and established, e.g. [1]. I can see the decomposition of the learnable variance schedule, and the regularized learning are novel. In Besides that, can the authors comment on the technical non-triviality of this extensio
The authors present an extension of VDMs to the conditional case, which has not yet been demonstrated and would indeed eliminate some hand-tuning when training conditional diffusion models. This is also well motivated by an interesting application to inverse problems in optics where uncertainty estimates would be very useful. The authors demonstrate that their learned schedules correspond well to more structured regions of their input on the super-resolution task.
While the authors motivate their work well, ~I have some concerns with the experiments performed and their presentation. I would be willing to raise my score if the following concerns are addressed.~ **Update**: The authors have addressed the major concerns, so I have updated my score accordingly. **Major concerns**: **Uncertainty results**: One of the main motivations of the paper is the ability to report uncertainty in order to point out artifacts in the solutions. In Figure 2, the authors
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced X-ray Imaging Techniques
MethodsDiffusion