Deep Gaussian Processes with Gradients

TL;DR

This paper introduces a Bayesian framework for Deep Gaussian Processes that incorporate gradient information, improving modeling of nonstationary functions and providing scalable inference methods.

Contribution

It presents a novel Bayesian approach for DGPs with gradients, including software implementation and benchmarking against existing models.

Findings

Enhanced nonstationary modeling with gradient information

Open-source 'deepgp' package available on CRAN

Demonstrated performance improvements over traditional DGPs and GPs with gradients

Abstract

Deep Gaussian processes (DGPs) are popular surrogate models for complex nonstationary computer experiments. DGPs use one or more latent Gaussian processes (GPs) to warp the input space into a plausibly stationary regime, then use typical GP regression on the warped domain. While this composition of GPs is conceptually straightforward, the functional nature of the multi-dimensional latent warping makes Bayesian posterior inference challenging. Traditional GPs with smooth kernels are naturally suited for the integration of gradient information, but the integration of gradients within a DGP presents new challenges and has yet to be explored. We propose a novel and comprehensive Bayesian framework for DGPs with gradients that facilitates both gradient-enhancement and gradient posterior predictive distributions. We provide open-source software in the "deepgp" package on CRAN, with optional Vecchia approximation to circumvent cubic computational bottlenecks. We benchmark our DGPs with gradients on a variety of nonstationary simulations, showing improvement over both GPs with gradients and conventional DGPs.