DualFast: Dual-Speedup Framework for Fast Sampling of Diffusion Models
Hu Yu, Hao Luo, Fan Wang, Feng Zhao
TL;DR
DualFast is a training-free framework that accelerates diffusion model sampling by disentangling and minimizing both discretization and approximation errors, significantly improving speed and quality in few-step sampling scenarios.
Contribution
It introduces a novel dual-error disentanglement strategy and a unified acceleration framework compatible with existing samplers, enhancing sampling speed and quality.
Findings
Significantly improves sampling speed in few-step scenarios.
Reduces total sampling error by addressing both discretization and approximation errors.
Compatible with various diffusion model samplers and domains.
Abstract
Diffusion probabilistic models (DPMs) have achieved impressive success in visual generation. While, they suffer from slow inference speed due to iterative sampling. Employing fewer sampling steps is an intuitive solution, but this will also introduces discretization error. Existing fast samplers make inspiring efforts to reduce discretization error through the adoption of high-order solvers, potentially reaching a plateau in terms of optimization. This raises the question: can the sampling process be accelerated further? In this paper, we re-examine the nature of sampling errors, discerning that they comprise two distinct elements: the widely recognized discretization error and the less explored approximation error. Our research elucidates the dynamics between these errors and the step by implementing a dual-error disentanglement strategy. Building on these foundations, we introduce an…
Peer Reviews
Decision·Submitted to ICLR 2025
1. The paper proposed an effective technique to improve samplers for diffusion models without introducing extra NFEs, especially in small NFE cases. The method can be readily combined with a wide range of existing samplers without post-training the diffusion models. 2. Extensive experiments were conducted to show the performance of the proposed method. Experiment settings vary across different condition types, sampler orders, guidance strategies, sampling space and datasets.
1. While the paper performed motivational experiments to explain discretization error and approximation error, the two types of error are not discriminated in a formal formulation. It is also not clear why the new noise estimation term in eq 10-12 can help reduce the approximation error. Here \tau is a variable time step and is set to T in experiments. The connection between the solver and the choice of \tau is not specified. 2. The writing of the paper may need improving. The notations in form
The paper’s idea is interesting, and the presentation is clear and easy to follow. Addressing discretization error is a important topic, and this work proposes a novel approach in that direction.
Several aspects need further clarification: - **Main Observation on Discretization Error:** The central observation, illustrated in Figure 2, is that discretization error decreases over time. The y-axis is labeled with MSE, could the authors clarify what quantity the MSE is measuring and how it directly relates to the discretization error? - **Interpretation of Figure 2:** The paper infers from Figure 2 that "the noise pattern is more recognizable as t increases,". But I feel there are oth
The method addresses a highly important issue (approximation error) in DPM sampling which they note is neglecting in prior works. Moreover, they demonstrate that this issue may have a greater impact than discretization error. Therefore, this is a highly relevant problem. The training-free approach to mitigate approximation error is a good contribution, especially given the fact that this approach can be used in a plug-and-play manner with many DPM sampling techniques addressing discretization e
Some aspects of the background and method are not very clear. For instance, in equations 8 and 10, are both x_\theta and \epsilon_\theta outputs of the neural network? It is not immediately clear to me how this improves the approximation error of the neural network. I think the main contribution should be made more clear, or at least, it should be more clear in the main text how the modified sampling scheme leads to decreased approximation error. I think the order of Sections 2.1 and 2.2 is a b
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsMedical Imaging Techniques and Applications · Advanced Neuroimaging Techniques and Applications · Advanced MRI Techniques and Applications