Sum of Squares Circuits
Lorenzo Loconte, Stefan Mengel, Antonio Vergari

TL;DR
This paper introduces sum of squares probabilistic circuits, a new class that enhances expressiveness and supports efficient inference, unifying and extending existing tractable models in probabilistic machine learning.
Contribution
The paper provides a theoretical characterization of sum of squares PCs, demonstrating their exponential expressiveness and establishing a hierarchy among tractable probabilistic models.
Findings
Sum of squares PCs are more expressive than squared and monotonic PCs.
Sum of squares PCs can unify various tractable models like Born Machines and PSD models.
Empirical results show sum of squares circuits excel in distribution estimation tasks.
Abstract
Designing expressive generative models that support exact and efficient inference is a core question in probabilistic ML. Probabilistic circuits (PCs) offer a framework where this tractability-vs-expressiveness trade-off can be analyzed theoretically. Recently, squared PCs encoding subtractive mixtures via negative parameters have emerged as tractable models that can be exponentially more expressive than monotonic PCs, i.e., PCs with positive parameters only. In this paper, we provide a more precise theoretical characterization of the expressiveness relationships among these models. First, we prove that squared PCs can be less expressive than monotonic ones. Second, we formalize a novel class of PCs -- sum of squares PCs -- that can be exponentially more expressive than both squared and monotonic PCs. Around sum of squares PCs, we build an expressiveness hierarchy that allows us to…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
Taxonomy
TopicsNeural Networks and Applications · Numerical Methods and Algorithms
