What's Inside Your Diffusion Model? A Score-Based Riemannian Metric to Explore the Data Manifold
Simone Azeglio, Arianna Di Bernardo
TL;DR
This paper introduces a score-based Riemannian metric for diffusion models to understand the geometry of the data manifold, enabling meaningful interpolation and extrapolation in image space.
Contribution
It proposes a novel Riemannian metric leveraging the Stein score function to characterize the data manifold without explicit parameterization, facilitating geodesic computation.
Findings
Score-based geodesics align with data manifold contours
Method outperforms baselines on LPIPS, FID, and KID metrics
Produces smoother, more realistic image transitions
Abstract
Recent advances in diffusion models have demonstrated their remarkable ability to capture complex image distributions, but the geometric properties of the learned data manifold remain poorly understood. We address this gap by introducing a score-based Riemannian metric that leverages the Stein score function from diffusion models to characterize the intrinsic geometry of the data manifold without requiring explicit parameterization. Our approach defines a metric tensor in the ambient space that stretches distances perpendicular to the manifold while preserving them along tangential directions, effectively creating a geometry where geodesics naturally follow the manifold's contours. We develop efficient algorithms for computing these geodesics and demonstrate their utility for both interpolation between data points and extrapolation beyond the observed data distribution. Through…
Peer Reviews
Decision·ICLR 2026 Conference Withdrawn Submission
- The paper introduces a simple and clear formulation of a score-based Riemannian metric, offering a geometric interpretation of diffusion models. - The proposed framework opens a promising direction for understanding the latent structure of diffusion models. - The writing is generally clear and easy to follow.
- The proposed metric $g(x) = I + \lambda s(x) s(x)^{T}$ penalizes only a single direction along the score vector $s(x)$. When the data manifold has a codimension greater than one, normal directions orthogonal to $s(x)$ remain unpenalized. This limitation raises questions about the completeness of the metric in capturing local manifold curvature. - The Geodesic Computations Algorithm requires heuristic techniques, such as adding the same noise in Line 223. Furthermore, the denoising step in Line
1. The method doesn't require additional training to obtain a metric on the data manifold given a pre-trained diffusion model. The authors also proposes an extrapolation method. 2. The method shows smoother interpolation and better performance than LERP/SLERP/NoiseDiff on synthetic sphere, Rotated-MNIST, and Stable Diffusion latents (MorphBench) 3. The authors shows some successful extrapolation examples
1. The paper proposes a method for defining a Riemannian metric on the data manifold for interpolation, but does not compare with many prior works that proposes methods for similar applications for data manifold interpolation (e.g. those listed in appendix A's related work section, https://arxiv.org/abs/2410.12779, GRAE, etc), especially methods that also learns a metric on the data manifold for interpolation (https://arxiv.org/abs/2405.14780, https://arxiv.org/abs/2410.04543). While these metho
The proposed approach is simple, intuitive, and theoretically interpretable, compared to existing studies that try to understand the internals of pretrained generative models. This paper clearly explains the implementation, which makes it run off-the-shelf. This paper presents both analytical results on synthetic data and practical results on real data.
### Soundness (1) This metric uses an extremely large coefficient $\lambda = 1000$, which makes $g$ nearly rank-1. This raises doubts about the validity as a Riemannian metric. When $\lambda$ is small, the method reduces to LERP and is expected to perform poorly. Also, there is a gap between the "deformation of the ambient space" described in Figure 1 and what is actually done. Relatedly, since g$(v, v) = <v, v> + \lambda <v, s>^2$, geodesics essentially minimize $<v, s>$. However, this vanishe
The paper is well written and provides interesting intuition about constructing a data-driven Riemannian metric that can explain the geometry of the data manifold. They show promising results in uncovering geodesics on simple low-dimensional hyperspheres isometrically embedded in high dimensions, and some improvement relative to linear interpolation on rotated MNIST.
1. **Theoretical justification.** In Section 3.2 the authors claim, “This energy functional penalizes curves that move in directions normal to the data manifold.” Prior work [3] shows that for small diffusion times the score lies in the normal space, but that does not imply the specific metric $G(x)=I+\lambda s(x)s(x)^\top$ will drive velocities to the tangent bundle. The curve velocity $\dot\gamma$ can still have substantial projection onto the normal space in directions orthogonal to $s(x)$. T
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsGenerative Adversarial Networks and Image Synthesis · Morphological variations and asymmetry · Advanced Neuroimaging Techniques and Applications
MethodsDiffusion