Influence of Prior Distributions on Gaussian Process Hyperparameter Inference

TL;DR

This paper investigates how different prior and proposal distributions impact the predictive accuracy and uncertainty quantification in Bayesian Gaussian process models, emphasizing the importance of prior choice in hyperparameter inference.

Contribution

It systematically analyzes the influence of prior and proposal distributions on Gaussian process hyperparameter inference using both simulated and real data.

Findings

Prior choices significantly affect predictive performance.

Proposal distribution selection impacts sampling efficiency.

Hierarchical Bayesian approaches improve uncertainty quantification.

Abstract

Gaussian processes (GPs) are widely used metamodels for approximating expensive computer simulations, particularly in engineering design and spatial prediction. However, their performance can deteriorate significantly when covariance parameters are poorly estimated, highlighting the importance of accurate inference. The most common approach involves maximizing the marginal likelihood, yielding point estimates of these parameters. However, this approach is highly sensitive to initialization and optimization settings. An alternative is to adopt a fully Bayesian hierarchical framework, where the posterior distribution over the covariance parameters is inferred. This approach provides more robust uncertainty quantification and reduces sensitivity to parameter selection. Yet, a key challenge lies in the careful specification of prior distributions for these parameters. While many available software packages provide default priors, their influence on model behavior is often underexplored. Additionally, the choice of proposal distributions can also influence sampling efficiency and convergence. In this paper, we examine how different prior and proposal distributions over the lengthscale parameters $\theta$ affect predictive performance in a hierarchical GP model, using both simulated and real data experiments. By evaluating various types of priors and proposals, we aim to better understand their influence on predictive accuracy and uncertainty quantification.