Random Sum-Product Forests with Residual Links

TL;DR

This paper introduces Random Sum-Product Forests (RSPFs), an ensemble of randomly generated SPNs with residual links, improving density estimation for complex data and achieving competitive performance with structure learning methods.

Contribution

The paper proposes RSPFs, a novel ensemble approach combining random SPNs with residual links, enhancing expressiveness and performance in density estimation tasks.

Findings

RSPFs outperform individual SPNs in density estimation.

Residual links further improve model performance.

ResSPNs are competitive with established structure learning methods.

Abstract

Tractable yet expressive density estimators are a key building block of probabilistic machine learning. While sum-product networks (SPNs) offer attractive inference capabilities, obtaining structures large enough to fit complex, high-dimensional data has proven challenging. In this paper, we present random sum-product forests (RSPFs), an ensemble approach for mixing multiple randomly generated SPNs. We also introduce residual links, which reference specialized substructures of other component SPNs in order to leverage the context-specific knowledge encoded within them. Our empirical evidence demonstrates that RSPFs provide better performance than their individual components. Adding residual links improves the models further, allowing the resulting ResSPNs to be competitive with commonly used structure learning methods.