A Unified Approach for Learning the Parameters of Sum-Product Networks

TL;DR

This paper introduces a unified framework for learning Sum-Product Networks (SPNs) parameters, demonstrating their equivalence to mixture models and proposing algorithms that improve learning efficiency and connect to EM.

Contribution

It provides a novel theoretical foundation for SPN parameter learning, formulates the problem as a signomial program, and develops two effective algorithms using SMA and CCCP.

Findings

Algorithms avoid projection operations with multiplicative updates

CCCP-based method aligns with EM for SPNs

Unified framework clarifies SPN learning as mixture modeling

Abstract

We present a unified approach for learning the parameters of Sum-Product networks (SPNs). We prove that any complete and decomposable SPN is equivalent to a mixture of trees where each tree corresponds to a product of univariate distributions. Based on the mixture model perspective, we characterize the objective function when learning SPNs based on the maximum likelihood estimation (MLE) principle and show that the optimization problem can be formulated as a signomial program. We construct two parameter learning algorithms for SPNs by using sequential monomial approximations (SMA) and the concave-convex procedure (CCCP), respectively. The two proposed methods naturally admit multiplicative updates, hence effectively avoiding the projection operation. With the help of the unified framework, we also show that, in the case of SPNs, CCCP leads to the same algorithm as Expectation Maximization (EM) despite the fact that they are different in general.