Polynomial Convergence of Riemannian Diffusion Models
Xingyu Xu, Ziyi Zhang, Yorie Nakahira, Guannan Qu, Yuejie Chi
TL;DR
This paper proves that Riemannian diffusion models can achieve accurate data sampling with polynomially small steps on curved spaces, under mild assumptions, improving theoretical understanding of non-Euclidean generative modeling.
Contribution
It establishes that polynomial step sizes are sufficient for small sampling error in Riemannian diffusion models with $L_2$-accurate scores, removing the need for smoothness or positivity assumptions.
Findings
Polynomial stepsize guarantees small total variation error.
Analysis applies under mild curvature conditions.
Advances understanding of diffusion models on curved manifolds.
Abstract
Diffusion models have demonstrated remarkable empirical success in the recent years and are considered one of the state-of-the-art generative models in modern AI. These models consist of a forward process, which gradually diffuses the data distribution to a noise distribution spanning the whole space, and a backward process, which inverts this transformation to recover the data distribution from noise. Most of the existing literature assumes that the underlying space is Euclidean. However, in many practical applications, the data are constrained to lie on a submanifold of Euclidean space. Addressing this setting, De Bortoli et al. (2022) introduced Riemannian diffusion models and proved that using an exponentially small step size yields a small sampling error in the Wasserstein distance, provided the data distribution is smooth and strictly positive, and the score estimate is…
Peer Reviews
Decision·ICLR 2026 Poster
- **Problem significance.** TV‑accurate sampling on manifolds with polynomial step size is a meaningful strengthening over prior Riemannian SGM bounds that scale poorly with $d$; cf. De Bortoli et al. 2022. - **Technique.** The parametrix approach to BM simulation error is original in this context and technically apt; it uses BGV-style local expansions and Volterra series to compare kernels; Berline-Getzler-Bergne (https://link.springer.com/book/10.1007/978-3-642-58088-8). - **Error decompositio
1. **Reset-analysis mismatch.** Provide a uniform upper bound on $$\sum^N_{k=1} \mathrm{Pr} (\| \Delta_k \| > h^{1/4}) $$ and propagate it into the final TV bound. One can combine drift bounds (from Li-Yau/Hamilton/Han-Zhang estimates) with a BDG-type tail on the martingale term to show a sub-Gaussian decay in $(\omega^2 / (t_k - t))$; but the constants and dependence on $d$, $K$, $\rho^{-1}$, and $\delta^{-1}$ must be explicit. 2. **Parametrix generator sign.** Unify the generator throughout §
The work has several notable strengths: 1. The presentation is clear and pedagogical, making the material accessible even to readers without a strong background in differential geometry. 2. The discussion of related work is thorough and well-integrated, situating the paper within a coherent and well-defined research landscape. 3. The analytical results appear sound, and the way of deriving the bound over the total variation, as a measure of the distance between two distributions, could be emp
The results are primarily analytical and the central goal of the work is to derive new performance bounds for RSGMs. I hence appreciate the total analytical vocation of the manuscript. On the other hand, one might think that the total absence of experimental validation could appear as a weakness of the work. De Bortoli et al. (2022) compare RSGM with other manifold-based diffusive routines on manifold supported datasets, like climate science spherical data. They quantify the quality of the perfo
- The paper presents the first polynomial TV convergence results for Riemannian diffusion models to the best of my knowledge. While it extends the theoretical results in Riemannian diffusion models, the paper proves TV convergence instead of Wasserstein convergence under mild geometric conditions. - The paper provides analytical techniques to prove the polynomial convergence: Li-Yau estimate for the heat kernel, localization of the drift fields, and parametrix estimates. These tools may be mea
- I'm unsure what the impact of the paper is and how it contributes to the field. While the paper gives a theoretical guarantee for the Riemannian diffusion model, the empirical results from De Bortoli et al already validated that it worked in diverse settings. I agree that the paper provided new tools to use for future study in this field, but it is unclear how important it is to show polynomial TV convergence for the already working model. If the paper presented a recipe for stepsize (with emp
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsStochastic Gradient Optimization Techniques · Generative Adversarial Networks and Image Synthesis · Geometric Analysis and Curvature Flows