Scaling Continuous Latent Variable Models as Probabilistic Integral Circuits

TL;DR

This paper introduces scalable methods for building and training DAG-shaped probabilistic integral circuits (PICs) with neural sharing, improving their efficiency and effectiveness over traditional probabilistic circuits.

Contribution

It presents a new pipeline for constructing DAG-shaped PICs, tensorized training procedures, and neural sharing techniques for scalable probabilistic modeling.

Findings

QPCs outperform traditional PCs in experiments

Neural sharing enhances scalability of PIC training

DAG-shaped PICs enable flexible variable decompositions

Abstract

Probabilistic integral circuits (PICs) have been recently introduced as probabilistic models enjoying the key ingredient behind expressive generative models: continuous latent variables (LVs). PICs are symbolic computational graphs defining continuous LV models as hierarchies of functions that are summed and multiplied together, or integrated over some LVs. They are tractable if LVs can be analytically integrated out, otherwise they can be approximated by tractable probabilistic circuits (PC) encoding a hierarchical numerical quadrature process, called QPCs. So far, only tree-shaped PICs have been explored, and training them via numerical quadrature requires memory-intensive processing at scale. In this paper, we address these issues, and present: (i) a pipeline for building DAG-shaped PICs out of arbitrary variable decompositions, (ii) a procedure for training PICs using tensorized circuit architectures, and (iii) neural functional sharing techniques to allow scalable training. In extensive experiments, we showcase the effectiveness of functional sharing and the superiority of QPCs over traditional PCs.