Gaussian Process Regression Networks

TL;DR

The paper introduces Gaussian process regression networks (GPRN), a flexible Bayesian model combining neural network structures with Gaussian processes, capable of modeling complex input-dependent correlations and heavy-tailed distributions.

Contribution

It presents a novel regression framework that integrates Bayesian neural networks with Gaussian processes, enabling input-dependent correlations and improved modeling of multivariate data.

Findings

Outperforms eight popular multi-task Gaussian process models on benchmark datasets.

Demonstrates significant improvements in multivariate volatility modeling.

Effective on high-dimensional gene expression data.

Abstract

We introduce a new regression framework, Gaussian process regression networks (GPRN), which combines the structural properties of Bayesian neural networks with the non-parametric flexibility of Gaussian processes. This model accommodates input dependent signal and noise correlations between multiple response variables, input dependent length-scales and amplitudes, and heavy-tailed predictive distributions. We derive both efficient Markov chain Monte Carlo and variational Bayes inference procedures for this model. We apply GPRN as a multiple output regression and multivariate volatility model, demonstrating substantially improved performance over eight popular multiple output (multi-task) Gaussian process models and three multivariate volatility models on benchmark datasets, including a 1000 dimensional gene expression dataset.