Latent Abstractions in Generative Diffusion Models
Giulio Franzese, Mattia Martini, Giulio Corallo, Paolo Papotti, Pietro, Michiardi
TL;DR
This paper introduces a new theoretical framework for understanding how diffusion models generate high-dimensional data by leveraging low-dimensional latent abstractions, supported by empirical validation.
Contribution
It extends existing theories with a novel formulation of joint dynamics and an information-theoretic measure, providing a new perspective on SDE-based generative models.
Findings
Diffusion models can be viewed as non-linear filters influenced by latent abstractions.
Latent abstractions emerge at different stages of the generative process.
The theory is empirically validated with experimental results.
Abstract
In this work we study how diffusion-based generative models produce high-dimensional data, such as an image, by implicitly relying on a manifestation of a low-dimensional set of latent abstractions, that guide the generative process. We present a novel theoretical framework that extends NLF, and that offers a unique perspective on SDE-based generative models. The development of our theory relies on a novel formulation of the joint (state and measurement) dynamics, and an information-theoretic measure of the influence of the system state on the measurement process. According to our theory, diffusion models can be cast as a system of SDE, describing a non-linear filter in which the evolution of unobservable latent abstractions steers the dynamics of an observable measurement process (corresponding to the generative pathways). In addition, we present an empirical study to validate our…
Peer Reviews
Decision·Submitted to ICLR 2025
They rigorously define the connection between nonlinear filtering and diffusion.
The paper asks the reader to spend hours understanding their notations and results before describing the motivation in section 4. The paper is very challenging to read as is and I strongly recommend the authors clearly and succinctly describe their contributions before the technical details, which should also mostly be moved to the appendix. While a connection between two mathematical frameworks is challenging to derive, it is only interesting if it leads to a deeper understanding of either te
* Well-motivated introduction, situating the work within existing literature and highlighting its contributions. * The structure is logically organized, and the mathematical formulations are rigorous, enhancing the soundness of the theory. * The proposed information-theoretic measure effectively captures changes in latent abstractions during generation, with experiments showing jumps in the measure that align with theoretical expectations.
I have no significant weaknesses to report. Please see questions for minor points of clarification.
The NLF viewpoint on diffusion model is to my knowledge new, and potentially interesting. The use of mutual information measurement is also helpful in quantifying the effect of noising/denoising on the latent variables (here labels). The experiments seem carefully carried out.
The methodology is sound provided that the datasets can be understood as being dictated by a low-dimensional set of latent variables, and that it is possible to understand the generative process as inferring the latent variables in a Baysian way. The experiments are also conducted solely on such a synthetic dataset, with each sample having a predetermined set of labels. Either the including of a theoretical discussion on this underlying basic assumption, or devising experiments on more diverse d
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsNatural Language Processing Techniques
MethodsSparse Evolutionary Training · Diffusion