Sum-Product Networks for Hybrid Domains

TL;DR

This paper introduces a novel deep probabilistic architecture called Mixed SPNs that efficiently models complex hybrid data distributions without requiring prior specification of variable types.

Contribution

It presents the first trainable deep probabilistic model for hybrid domains using Sum-Product Networks with nonparametric decomposition, enabling flexible and efficient learning and inference.

Findings

Effective in modeling complex hybrid distributions

Handles a wide range of continuous and discrete variables

Facilitates easier probabilistic modeling in diverse domains

Abstract

While all kinds of mixed data -from personal data, over panel and scientific data, to public and commercial data- are collected and stored, building probabilistic graphical models for these hybrid domains becomes more difficult. Users spend significant amounts of time in identifying the parametric form of the random variables (Gaussian, Poisson, Logit, etc.) involved and learning the mixed models. To make this difficult task easier, we propose the first trainable probabilistic deep architecture for hybrid domains that features tractable queries. It is based on Sum-Product Networks (SPNs) with piecewise polynomial leave distributions together with novel nonparametric decomposition and conditioning steps using the Hirschfeld-Gebelein-R\'enyi Maximum Correlation Coefficient. This relieves the user from deciding a-priori the parametric form of the random variables but is still expressive enough to effectively approximate any continuous distribution and permits efficient learning and inference. Our empirical evidence shows that the architecture, called Mixed SPNs, can indeed capture complex distributions across a wide range of hybrid domains.