Gaussian Process Networks

TL;DR

This paper introduces Gaussian Process Networks, a semi-parametric Bayesian approach for learning the structure of continuous-variable networks, capable of modeling complex nonlinear relationships and computing marginal likelihoods directly.

Contribution

It proposes a novel family of probabilistic networks based on Gaussian Process priors, enabling direct marginal likelihood computation for structure learning in continuous domains.

Findings

Successfully learned nonlinear dependencies in artificial data

Demonstrated effectiveness on real-life datasets with complex relations

Outperformed traditional Gaussian-based methods in capturing functional dependencies

Abstract

In this paper we address the problem of learning the structure of a Bayesian network in domains with continuous variables. This task requires a procedure for comparing different candidate structures. In the Bayesian framework, this is done by evaluating the {em marginal likelihood/} of the data given a candidate structure. This term can be computed in closed-form for standard parametric families (e.g., Gaussians), and can be approximated, at some computational cost, for some semi-parametric families (e.g., mixtures of Gaussians). We present a new family of continuous variable probabilistic networks that are based on {em Gaussian Process/} priors. These priors are semi-parametric in nature and can learn almost arbitrary noisy functional relations. Using these priors, we can directly compute marginal likelihoods for structure learning. The resulting method can discover a wide range of functional dependencies in multivariate data. We develop the Bayesian score of Gaussian Process Networks and describe how to learn them from data. We present empirical results on artificial data as well as on real-life domains with non-linear dependencies.