Chained Gaussian Processes

TL;DR

This paper introduces Chained Gaussian Processes, a flexible extension of Gaussian process models that allows for non-linear combinations of likelihood parameters, with a scalable inference method applicable to various likelihoods.

Contribution

It proposes a novel class of models called Chained Gaussian Processes and develops an approximate inference method for them, overcoming the limitations of traditional link functions.

Findings

The inference method is scalable and applicable to any factorized likelihood.

Demonstrated effectiveness on a range of likelihood functions.

Allows modeling with non-linear combinations of likelihood parameters.

Abstract

Gaussian process models are flexible, Bayesian non-parametric approaches to regression. Properties of multivariate Gaussians mean that they can be combined linearly in the manner of additive models and via a link function (like in generalized linear models) to handle non-Gaussian data. However, the link function formalism is restrictive, link functions are always invertible and must convert a parameter of interest to a linear combination of the underlying processes. There are many likelihoods and models where a non-linear combination is more appropriate. We term these more general models Chained Gaussian Processes: the transformation of the GPs to the likelihood parameters will not generally be invertible, and that implies that linearisation would only be possible with multiple (localized) links, i.e. a chain. We develop an approximate inference procedure for Chained GPs that is scalable and applicable to any factorized likelihood. We demonstrate the approximation on a range of likelihood functions.