Modular Jump Gaussian Processes

TL;DR

This paper introduces a modular approach to jump Gaussian processes that improves modeling of non-stationary data with sudden changes by avoiding complex joint inference, using local neighborhoods and cluster-based features.

Contribution

It proposes a simplified, modular framework for jump Gaussian processes that retains core ideas but enhances practicality and performance without joint inference.

Findings

Significant improvements in modeling jump processes.

Effective neighborhood size learning respects data discontinuities.

Cluster-based features capture distinct output regions.

Abstract

Gaussian processes (GPs) furnish accurate nonlinear predictions with well-calibrated uncertainty. However, the typical GP setup has a built-in stationarity assumption, making it ill-suited for modeling data from processes with sudden changes, or "jumps" in the output variable. The "jump GP" (JGP) was developed for modeling data from such processes, combining local GPs and latent "level" variables under a joint inferential framework. But joint modeling can be fraught with difficulty. We aim to simplify by suggesting a more modular setup, eschewing joint inference but retaining the main JGP themes: (a) learning optimal neighborhood sizes that locally respect manifolds of discontinuity; and (b) a new cluster-based (latent) feature to capture regions of distinct output levels on both sides of the manifold. We show that each of (a) and (b) separately leads to dramatic improvements when modeling processes with jumps. In tandem (but without requiring joint inference) that benefit is compounded, as illustrated on real and synthetic benchmark examples from the recent literature.