Numerical issues in maximum likelihood parameter estimation for Gaussian process interpolation

TL;DR

This paper examines numerical problems in maximum likelihood estimation for Gaussian process interpolation and proposes strategies to improve the robustness of open-source GP software, which is vital for reliable Bayesian optimization studies.

Contribution

It identifies the sources of numerical issues in GP parameter estimation and offers practical solutions to enhance the stability of existing open-source implementations.

Findings

Numerical issues can significantly affect GP interpolation accuracy.

Simple strategies can improve the stability of GP parameter estimation.

Enhanced implementations lead to more reliable Bayesian optimization results.

Abstract

This article investigates the origin of numerical issues in maximum likelihood parameter estimation for Gaussian process (GP) interpolation and investigates simple but effective strategies for improving commonly used open-source software implementations. This work targets a basic problem but a host of studies, particularly in the literature of Bayesian optimization, rely on off-the-shelf GP implementations. For the conclusions of these studies to be reliable and reproducible, robust GP implementations are critical.