Observation-Guided Diffusion Probabilistic Models
Junoh Kang, Jinyoung Choi, Sungik Choi, Bohyung Han
TL;DR
This paper introduces Observation-Guided Diffusion Models (OGDM), a new image generation method that improves quality and sampling speed by integrating observation guidance into the diffusion process with a novel training objective.
Contribution
The paper presents a new training scheme for diffusion models that incorporates observation guidance, enhancing performance without extra inference cost.
Findings
Improved image quality with fewer function evaluations.
Compatible with various fast inference strategies.
Enhanced denoising networks through the proposed training method.
Abstract
We propose a novel diffusion-based image generation method called the observation-guided diffusion probabilistic model (OGDM), which effectively addresses the tradeoff between quality control and fast sampling. Our approach reestablishes the training objective by integrating the guidance of the observation process with the Markov chain in a principled way. This is achieved by introducing an additional loss term derived from the observation based on a conditional discriminator on noise level, which employs a Bernoulli distribution indicating whether its input lies on the (noisy) real manifold or not. This strategy allows us to optimize the more accurate negative log-likelihood induced in the inference stage especially when the number of function evaluations is limited. The proposed training scheme is also advantageous even when incorporated only into the fine-tuning process, and it is…
Peer Reviews
Decision·ICLR 2024 Conference Withdrawn Submission
1. The paper is well written and clear. 2. The paper presents detailed theoretic analysis for the proposed method. 3. Experiments on several datasets show the effectiveness of the proposed method.
1. The paper introduces additional cost for training, but there is no additional training cost analysis. 2. The advantage of diffusion models compared to GAN is the training stability, it introduce GAN training again, which may harm the training stability. 3. The experiments are conducted on unconditional generation, leaving its performance on the mainstream text-to-image generation models unclear.
1. The paper proposes a new training objective for diffusion models that is theoretically grounded. 2. The paper did comprehensive experiments on three datasets of various resolutions, using various sampling algorithms.
1. The proposed training pipeline is coupled with a specific sampling algorithm. At inference time, when the sampling algorithm is changed to another one that is different from the one used during training, the proposed method has limited improvements compared to baselines, as demonstrated in Table 3. This limits the applicability of the method, since if new sampling algorithm is proposed, the diffusion model also needs to be re-trained. 2. Comparison with important baselines are missing. Specif
* The authors' proposed method OGDM is novel to my knowledge. There were existing works that also explored utilizing a discriminator to help diffusion model training/sampling, but this paper's proposal is different from those. * Empirically, the proposed method demonstrates nontrivial benefits when incorporated into various baselines, especially with few NFEs.
1. There is no empirical comparison between the proposed method and the existing diffusion works that utilize a discriminator. Currently, the paper only compares with the "vanilla" diffusion training baselines. As we have already seen from prior works, incorporating a GAN component into diffusion models could improve the empirical results by a lot, I would highly suggest the authors do so, e.g. [1], [2], to give a clear picture of where this method stands against its peers. 2. The presentation o
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsGenerative Adversarial Networks and Image Synthesis · Gaussian Processes and Bayesian Inference · Domain Adaptation and Few-Shot Learning
MethodsDiffusion