Building Expressive and Tractable Probabilistic Generative Models: A Review

TL;DR

This paper surveys recent advancements in probabilistic circuits, emphasizing the balance between expressivity and tractability, and discusses future research directions in deep and hybrid models.

Contribution

It provides a comprehensive taxonomy and unified perspective on design principles, extensions, and challenges in tractable probabilistic generative modeling.

Findings

Unified taxonomy of probabilistic circuits

Insights into expressivity-tractability trade-offs

Discussion of deep and hybrid probabilistic models

Abstract

We present a comprehensive survey of the advancements and techniques in the field of tractable probabilistic generative modeling, primarily focusing on Probabilistic Circuits (PCs). We provide a unified perspective on the inherent trade-offs between expressivity and tractability, highlighting the design principles and algorithmic extensions that have enabled building expressive and efficient PCs, and provide a taxonomy of the field. We also discuss recent efforts to build deep and hybrid PCs by fusing notions from deep neural models, and outline the challenges and open questions that can guide future research in this evolving field.