Hierarchical shrinkage Gaussian processes: applications to computer code emulation and dynamical system recovery

TL;DR

This paper introduces a hierarchical shrinkage Gaussian process model that captures structured sparsity in response surfaces, improving predictions and feature identification in complex scientific applications.

Contribution

The paper proposes a novel HierGP model incorporating cumulative shrinkage priors, embedding effect sparsity, heredity, and hierarchy principles for better structured feature analysis.

Findings

HierGP outperforms existing models in numerical experiments.

Efficient posterior sampling algorithms are developed for HierGP.

The model demonstrates improved dynamical system recovery.

Abstract

In many areas of science and engineering, computer simulations are widely used as proxies for physical experiments, which can be infeasible or unethical. Such simulations can often be computationally expensive, and an emulator can be trained to efficiently predict the desired response surface. A widely-used emulator is the Gaussian process (GP), which provides a flexible framework for efficient prediction and uncertainty quantification. Standard GPs, however, do not capture structured sparsity on the underlying response surface, which is present in many applications, particularly in the physical sciences. We thus propose a new hierarchical shrinkage GP (HierGP), which incorporates such structure via cumulative shrinkage priors within a GP framework. We show that the HierGP implicitly embeds the well-known principles of effect sparsity, heredity and hierarchy for analysis of experiments, which allows our model to identify structured sparse features from the response surface with limited data. We propose efficient posterior sampling algorithms for model training and prediction, and prove desirable consistency properties for the HierGP. Finally, we demonstrate the improved performance of HierGP over existing models, in a suite of numerical experiments and an application to dynamical system recovery.