On the Relationship Between Monotone and Squared Probabilistic Circuits

TL;DR

This paper explores the relationship between monotone and squared probabilistic circuits, introduces Inception PCs that unify both approaches, and demonstrates their superior performance on various datasets.

Contribution

It proposes Inception PCs, a novel circuit model that unifies monotone and squared circuits using complex parameters, enhancing modeling flexibility.

Findings

Inception PCs outperform both monotone and squared circuits on tabular datasets.

Inception PCs show improved accuracy on image datasets.

The model demonstrates the theoretical unification of two circuit types.

Abstract

Probabilistic circuits are a unifying representation of functions as computation graphs of weighted sums and products. Their primary application is in probabilistic modeling, where circuits with non-negative weights (monotone circuits) can be used to represent and learn density/mass functions, with tractable marginal inference. Recently, it was proposed to instead represent densities as the square of the circuit function (squared circuits); this allows the use of negative weights while retaining tractability, and can be exponentially more expressive efficient than monotone circuits. Unfortunately, we show the reverse also holds, meaning that monotone circuits and squared circuits are incomparable in general. This raises the question of whether we can reconcile, and indeed improve upon the two modeling approaches. We answer in the positive by proposing Inception PCs, a novel type of circuit that naturally encompasses both monotone circuits and squared circuits as special cases, and employs complex parameters. Empirically, we validate that Inception PCs can outperform both monotone and squared circuits on a range of tabular and image datasets.