Sequential construction and dimension reduction of Gaussian processes under constraints

TL;DR

This paper introduces the MaxMod algorithm for Gaussian process modeling that efficiently performs dimension reduction and knot allocation under inequality constraints, enabling scalable modeling in higher dimensions up to at least 20.

Contribution

The paper presents the MaxMod algorithm, which combines dimension reduction and knot placement in constrained Gaussian processes, with proven convergence and improved scalability.

Findings

MaxMod remains efficient in higher dimensions (up to at least 20).

Requires fewer knots than existing constrained GP models for similar accuracy.

Proven convergence of the algorithm and the finite-dimensional GP extension.

Abstract

Accounting for inequality constraints, such as boundedness, monotonicity or convexity, is challenging when modeling costly-to-evaluate black box functions. In this regard, finite-dimensional Gaussian process (GP) regression models bring a valuable solution, as they guarantee that the inequality constraints are satisfied everywhere. Nevertheless, these models are currently restricted to small dimensional situations (up to dimension 5). Addressing this issue, we introduce the MaxMod algorithm that sequentially inserts one-dimensional knots or adds active variables, thereby performing at the same time dimension reduction and efficient knot allocation. We prove the convergence of this algorithm. In intermediary steps of the proof, we propose the notion of multi-affine extension and study its properties. We also prove the convergence of finite-dimensional GPs, when the knots are not dense in the input space, extending the recent literature. With simulated and real data, we demonstrate that the MaxMod algorithm remains efficient in higher dimension (at least in dimension 20), and needs fewer knots than other constrained GP models from the state-of-the-art, to reach a given approximation error.