Adaptive finite element type decomposition of Gaussian processes

TL;DR

This paper compares finite element and lattice-based Gaussian process approximations, showing that the lattice approach with adaptive basis selection achieves optimal convergence rates for various smoothness levels.

Contribution

It introduces an adaptive basis selection method for Gaussian process approximation that achieves rate-optimal convergence across different smoothness levels.

Findings

Lattice-based approach with adaptive basis selection is rate-optimal.

SPDE-based approach with fixed smoothness is suboptimal.

Efficient computational strategies are developed and validated.

Abstract

In this paper, we investigate a class of approximate Gaussian processes (GP) obtained by taking a linear combination of compactly supported basis functions with the basis coefficients endowed with a dependent Gaussian prior distribution. This general class includes a popular approach that uses a finite element approximation of the stochastic partial differential equation (SPDE) associated with Mat\'ern GP. We explored another scalable alternative popularly used in the computer emulation literature where the basis coefficients at a lattice are drawn from a Gaussian process with an inverse-Gamma bandwidth. For both approaches, we study concentration rates of the posterior distribution. We demonstrated that the SPDE associated approach with a fixed smoothness parameter leads to a suboptimal rate despite how the number of basis functions and bandwidth are chosen when the underlying true function is sufficiently smooth. On the flip side, we showed that the later approach is rate-optimal adaptively over all smoothness levels of the underlying true function if an appropriate prior is placed on the number of basis functions. Efficient computational strategies are developed and numerics are provided to illustrate the theoretical results.