TL;DR
This paper investigates the instability in diffusion ODEs used for image reconstruction, revealing how intrinsic properties cause errors to amplify, especially in high-dimensional data, and providing theoretical and experimental insights.
Contribution
It identifies and explains the intrinsic instability in PF-ODEs for diffusion reconstruction, linking it to data sparsity and high-dimensional effects, which was previously not well understood.
Findings
Instability can significantly amplify reconstruction errors.
The probability of instability approaches one as data dimensionality increases.
Experimental validation on diffusion models confirms the theoretical analysis.
Abstract
Diffusion reconstruction plays a critical role in various applications such as image editing, restoration, and style transfer. In theory, the reconstruction should be simple - it just inverts and regenerates images by numerically solving the Probability Flow-Ordinary Differential Equation (PF-ODE). Yet in practice, noticeable reconstruction errors have been observed, which cannot be well explained by numerical errors. In this work, we identify a deeper intrinsic property in the PF-ODE generation process, the instability, that can further amplify the reconstruction errors. The root of this instability lies in the sparsity inherent in the generation distribution, which means that the probability is concentrated on scattered and small regions while the vast majority remains almost empty. To demonstrate the existence of instability and its amplification on reconstruction error, we conduct…
Peer Reviews
Decision·Submitted to ICLR 2026
- The paper provides a definition of instability and analyzes its influence on reconstruction errors using both experimental and theoretical analyses.
The primary weakness of this paper lies in the lack of a clear, reasonable connection between the so-called “instability” and “reconstruction error” observed in diffusion models. Moreover, much of the discussion appears casual, with insufficient references, weak theoretical grounding, and inaccurate mathematical descriptions. - This paper attempts to explain reconstruction errors beyond numerical discretization error, attributing them to the concept of “instability” caused by the sparsity of th
1. The paper attempts to rigorously understand the instability phenomenon via a mathematical treatment. 2. The visualizations (mostly given in the Appendix) are convincing of the issue. 3. To my knowledge, the analysis on instability as a function of dimensionality is novel. However, I have not carefully checked the proofs for correctness.
1. Related work should be more carefully discussed. The instability phenomenon has been previously studied for general invertible networks (e.g., see [1]) and it is well understood that non volume perserving mappings are inherently more prone to this. In this sense, the novel contribution of the paper is unclear. 2. The writing is imprecise. For example: - The term "sparsity" is used to mean, as I understand, support on low-dimensional manifolds. This leads to confusing statements like: "t
1. This paper tries to solve an important problem, which is the error in diffusion model reconstruction. This is important for many applications and many papers have studied this. 2. Figure 4 shows experimentally that the reconstruction error and the instability coefficient have a positive correlation. This supports the paper's claim.
1. Lack of Contribution (Obvious Claims): The main claim of the paper, "an unstable system (function with large Jacobian norm) amplifies input noise (numerical error)," is a very well-known and obvious. The paper names this 'instability', but it does not persuasively show why this is a new contribution in the context of diffusion models. Figure 4, which shows a correlation, also just re-confirms this obvious result. 2. The Propositions are very weak to support the logic. Proposition 2.1 is an a
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
