Gaussian Process Regression and Classification under Mathematical Constraints with Learning Guarantees

TL;DR

This paper introduces constrained Gaussian processes (CGP), enabling the incorporation of mathematical constraints into Gaussian process models with closed-form solutions, maintaining optimal theoretical properties for regression and classification.

Contribution

The paper presents CGP, a novel Gaussian process model that incorporates constraints with closed-form densities and retains Gaussian process properties for improved learning guarantees.

Findings

CGP allows easy incorporation of constraints like non-negativity and monotonicity.

CGP's posterior distributions are also CGPs with closed-form expressions.

CGP inherits optimal posterior contraction rates from standard Gaussian processes.

Abstract

We introduce constrained Gaussian process (CGP), a Gaussian process model for random functions that allows easy placement of mathematical constrains (e.g., non-negativity, monotonicity, etc) on its sample functions. CGP comes with closed-form probability density function (PDF), and has the attractive feature that its posterior distributions for regression and classification are again CGPs with closed-form expressions. Furthermore, we show that CGP inherents the optimal theoretical properties of the Gaussian process, e.g. rates of posterior contraction, due to the fact that CGP is an Gaussian process with a more efficient model space.