Probabilistic Deep Learning using Random Sum-Product Networks

TL;DR

This paper introduces RAT-SPNs, a simplified probabilistic deep learning model using random sum-product network structures trained with standard deep learning techniques, offering accurate predictions with well-calibrated uncertainties.

Contribution

It proposes a novel approach to training sum-product networks with random structures using classical deep learning methods, enhancing their practicality and interpretability.

Findings

RAT-SPNs achieve prediction accuracy comparable to deep neural networks.

They provide well-calibrated uncertainties and are robust to missing data.

The models can effectively detect outliers and unusual samples.

Abstract

The need for consistent treatment of uncertainty has recently triggered increased interest in probabilistic deep learning methods. However, most current approaches have severe limitations when it comes to inference, since many of these models do not even permit to evaluate exact data likelihoods. Sum-product networks (SPNs), on the other hand, are an excellent architecture in that regard, as they allow to efficiently evaluate likelihoods, as well as arbitrary marginalization and conditioning tasks. Nevertheless, SPNs have not been fully explored as serious deep learning models, likely due to their special structural requirements, which complicate learning. In this paper, we make a drastic simplification and use random SPN structures which are trained in a "classical deep learning manner", i.e. employing automatic differentiation, SGD, and GPU support. The resulting models, called RAT-SPNs, yield prediction results comparable to deep neural networks, while still being interpretable as generative model and maintaining well-calibrated uncertainties. This property makes them highly robust under missing input features and enables them to naturally detect outliers and peculiar samples.