Deep Jump Gaussian Processes for Surrogate Modeling of High-Dimensional Piecewise Continuous Functions

TL;DR

This paper introduces Deep Jump Gaussian Processes (DJGP), a new scalable surrogate modeling method for high-dimensional, piecewise continuous functions that combines local linear projections with Jump Gaussian Processes to improve accuracy and uncertainty quantification.

Contribution

The paper proposes DJGP, integrating region-specific projections with JGPs, and develops a scalable variational inference algorithm, advancing high-dimensional surrogate modeling techniques.

Findings

DJGP achieves superior predictive accuracy over existing methods.

DJGP provides more reliable uncertainty quantification.

Theoretical analysis includes an oracle error bound for DJGP.

Abstract

We introduce Deep Jump Gaussian Processes (DJGP), a novel method for surrogate modeling of a piecewise continuous function on a high-dimensional domain. DJGP addresses the limitations of conventional Jump Gaussian Processes (JGP) in high-dimensional input spaces by integrating region-specific, locally linear projections with JGP modeling. These projections employ region-dependent matrices to capture local low-dimensional subspace structures, making them well suited to the inherently localized modeling behavior of JGPs, a variant of local Gaussian processes. To control model complexity, we place a Gaussian Process prior on the projection matrices, allowing them to evolve smoothly across the input space. The projected inputs are then modeled with a JGP to capture piecewise continuous relationships with the response. This yields a distinctive two-layer deep learning of GP/JGP. We further develop a scalable variational inference algorithm to jointly learn the projection matrices and JGP hyperparameters. Rigorous theoretical analysis and extensive empirical studies are provided to justify the proposed approach. In particular, we derive an oracle error bound for DJGP and decompose it into four distinct sources of error, which are then linked to practical implications. Experiments on synthetic and benchmark datasets demonstrate that DJGP achieves superior predictive accuracy and more reliable uncertainty quantification compared with existing methods.