Mercer kernels and integrated variance experimental design: connections between Gaussian process regression and polynomial approximation

TL;DR

This paper explores the connection between Gaussian process regression and polynomial approximation through optimal experimental design, introducing IVAR-based algorithms for improved surrogate modeling on complex domains.

Contribution

It develops gradient-based algorithms for IVAR-optimal design in GP regression and links GP with polynomial approximation via Mercer kernels, highlighting design impacts.

Findings

IVAR-based designs outperform entropy-based designs in GP regression.

GP and polynomial approximation coincide under specific kernel and design conditions.

Adaptive GP + IVAR methods excel for certain target functions.

Abstract

This paper examines experimental design procedures used to develop surrogates of computational models, exploring the interplay between experimental designs and approximation algorithms. We focus on two widely used approximation approaches, Gaussian process (GP) regression and non-intrusive polynomial approximation. First, we introduce algorithms for minimizing a posterior integrated variance (IVAR) design criterion for GP regression. Our formulation treats design as a continuous optimization problem that can be solved with gradient-based methods on complex input domains, without resorting to greedy approximations. We show that minimizing IVAR in this way yields point sets with good interpolation properties, and that it enables more accurate GP regression than designs based on entropy minimization or mutual information maximization. Second, using a Mercer kernel/eigenfunction perspective on GP regression, we identify conditions under which GP regression coincides with pseudospectral polynomial approximation. Departures from these conditions can be understood as changes either to the kernel or to the experimental design itself. We then show how IVAR-optimal designs, while sacrificing discrete orthogonality of the kernel eigenfunctions, can yield lower approximation error than orthogonalizing point sets. Finally, we compare the performance of adaptive Gaussian process regression and adaptive pseudospectral approximation for several classes of target functions, identifying features that are favorable to the GP + IVAR approach.