Top-Down Bayesian Posterior Sampling for Sum-Product Networks

TL;DR

This paper introduces a Bayesian sampling method for sum-product networks that significantly accelerates learning and enhances predictive accuracy, making it feasible for large-scale, real-time applications.

Contribution

It develops a novel Gibbs sampling approach with a new full conditional probability for efficient Bayesian learning in large-scale SPNs.

Findings

Sampling speed improved by 10 to 100 times

Achieved superior predictive performance across 20 datasets

Reduced learning-time complexity for large SPNs

Abstract

Sum-product networks (SPNs) are probabilistic models characterized by exact and fast evaluation of fundamental probabilistic operations. Its superior computational tractability has led to applications in many fields, such as machine learning with time constraints or accuracy requirements and real-time systems. The structural constraints of SPNs supporting fast inference, however, lead to increased learning-time complexity and can be an obstacle to building highly expressive SPNs. This study aimed to develop a Bayesian learning approach that can be efficiently implemented on large-scale SPNs. We derived a new full conditional probability of Gibbs sampling by marginalizing multiple random variables to expeditiously obtain the posterior distribution. The complexity analysis revealed that our sampling algorithm works efficiently even for the largest possible SPN. Furthermore, we proposed a hyperparameter tuning method that balances the diversity of the prior distribution and optimization efficiency in large-scale SPNs. Our method has improved learning-time complexity and demonstrated computational speed tens to more than one hundred times faster and superior predictive performance in numerical experiments on more than 20 datasets.