Monte Carlo Implementation of Gaussian Process Models for Bayesian Regression and Classification

TL;DR

This paper discusses implementing Gaussian process models for Bayesian regression and classification using Monte Carlo methods, enabling flexible modeling of complex data with hyperparameter sampling.

Contribution

It introduces Monte Carlo techniques for Gaussian process models, including hyperparameter sampling and extensions to t-distributed noise and classification models.

Findings

Feasible implementation for datasets up to 1000 cases

Effective hyperparameter sampling via Markov chain methods

Models can identify relevant input features

Abstract

Gaussian processes are a natural way of defining prior distributions over functions of one or more input variables. In a simple nonparametric regression problem, where such a function gives the mean of a Gaussian distribution for an observed response, a Gaussian process model can easily be implemented using matrix computations that are feasible for datasets of up to about a thousand cases. Hyperparameters that define the covariance function of the Gaussian process can be sampled using Markov chain methods. Regression models where the noise has a t distribution and logistic or probit models for classification applications can be implemented by sampling as well for latent values underlying the observations. Software is now available that implements these methods using covariance functions with hierarchical parameterizations. Models defined in this way can discover high-level properties of the data, such as which inputs are relevant to predicting the response.