From Classical Probabilistic Latent Variable Models to Modern Generative AI: A Unified Perspective

TL;DR

This paper offers a unified probabilistic perspective on the evolution of generative AI models, connecting classical latent variable models with modern deep generative architectures to clarify their shared principles and differences.

Contribution

It provides a comprehensive framework that links classical and contemporary generative models within the probabilistic latent variable model paradigm, enhancing understanding of their relationships and guiding future research.

Findings

Unified probabilistic taxonomy of generative models

Clarification of shared principles and differences among architectures

Conceptual roadmap for future generative AI development

Abstract

From large language models to multi-modal agents, Generative Artificial Intelligence (AI) now underpins state-of-the-art systems. Despite their varied architectures, many share a common foundation in probabilistic latent variable models (PLVMs), where hidden variables explain observed data for density estimation, latent reasoning, and structured inference. This paper presents a unified perspective by framing both classical and modern generative methods within the PLVM paradigm. We trace the progression from classical flat models such as probabilistic PCA, Gaussian mixture models, latent class analysis, item response theory, and latent Dirichlet allocation, through their sequential extensions including Hidden Markov Models, Gaussian HMMs, and Linear Dynamical Systems, to contemporary deep architectures: Variational Autoencoders as Deep PLVMs, Normalizing Flows as Tractable PLVMs, Diffusion Models as Sequential PLVMs, Autoregressive Models as Explicit Generative Models, and Generative Adversarial Networks as Implicit PLVMs. Viewing these architectures under a common probabilistic taxonomy reveals shared principles, distinct inference strategies, and the representational trade-offs that shape their strengths. We offer a conceptual roadmap that consolidates generative AI's theoretical foundations, clarifies methodological lineages, and guides future innovation by grounding emerging architectures in their probabilistic heritage.