Mix-nets: Factored Mixtures of Gaussians in Bayesian Networks With Mixed Continuous And Discrete Variables

TL;DR

This paper introduces Mix-nets, a Bayesian network model that combines mixtures of Gaussians over variable subsets to effectively model complex dependencies among mixed continuous and discrete variables, learned efficiently from data.

Contribution

It proposes a novel Bayesian network framework that integrates low-dimensional Gaussian mixtures for mixed variable types and introduces efficient algorithms for automatic learning from data.

Findings

Effective modeling of real scientific data

High accuracy in synthetic data modeling

Efficient learning algorithms demonstrated

Abstract

Recently developed techniques have made it possible to quickly learn accurate probability density functions from data in low-dimensional continuous space. In particular, mixtures of Gaussians can be fitted to data very quickly using an accelerated EM algorithm that employs multiresolution kd-trees (Moore, 1999). In this paper, we propose a kind of Bayesian networks in which low-dimensional mixtures of Gaussians over different subsets of the domain's variables are combined into a coherent joint probability model over the entire domain. The network is also capable of modeling complex dependencies between discrete variables and continuous variables without requiring discretization of the continuous variables. We present efficient heuristic algorithms for automatically learning these networks from data, and perform comparative experiments illustrated how well these networks model real scientific data and synthetic data. We also briefly discuss some possible improvements to the networks, as well as possible applications.