A Variational Approximation for Bayesian Networks with Discrete and Continuous Latent Variables

TL;DR

This paper introduces a variational approximation method for Bayesian networks with mixed discrete and continuous variables, enabling faster and more accurate inference compared to sampling methods.

Contribution

It presents a novel variational approach to approximate the logistic function, improving inference efficiency and accuracy in complex Bayesian networks with discrete and continuous nodes.

Findings

Faster inference than sampling methods.

Potentially more accurate approximation.

Effective handling of evidence with arbitrary distributions.

Abstract

We show how to use a variational approximation to the logistic function to perform approximate inference in Bayesian networks containing discrete nodes with continuous parents. Essentially, we convert the logistic function to a Gaussian, which facilitates exact inference, and then iteratively adjust the variational parameters to improve the quality of the approximation. We demonstrate experimentally that this approximation is faster and potentially more accurate than sampling. We also introduce a simple new technique for handling evidence, which allows us to handle arbitrary distributions on observed nodes, as well as achieving a significant speedup in networks with discrete variables of large cardinality.