Deep generative models as the probability transformation functions

TL;DR

This paper presents a unified theoretical framework viewing all deep generative models as probability transformation functions, enabling cross-architecture methodological transfer and foundational understanding for improved generative modeling.

Contribution

It introduces a unifying perspective that conceptualizes diverse generative models as probability transformation functions, fostering theoretical development and methodological improvements.

Findings

All deep generative models fundamentally transform simple distributions into complex data distributions.

This perspective unifies various architectures like GANs, VAEs, flows, and diffusion models.

It enables transfer of techniques and theoretical insights across different generative model types.

Abstract

This paper introduces a unified theoretical perspective that views deep generative models as probability transformation functions. Despite the apparent differences in architecture and training methodologies among various types of generative models - autoencoders, autoregressive models, generative adversarial networks, normalizing flows, diffusion models, and flow matching - we demonstrate that they all fundamentally operate by transforming simple predefined distributions into complex target data distributions. This unifying perspective facilitates the transfer of methodological improvements between model architectures and provides a foundation for developing universal theoretical approaches, potentially leading to more efficient and effective generative modeling techniques.