Deep State-Space Gaussian Processes

TL;DR

This paper introduces a state-space approach to deep Gaussian process regression, representing the hierarchical model as a system of SDEs, enabling efficient sequential estimation and application to real-world signals like gravitational waves.

Contribution

It develops a novel state-space formulation for deep Gaussian processes, allowing scalable inference via sequential methods and demonstrating practical applications.

Findings

Linear computational complexity with respect to data size

Effective detection of gravitational waves from LIGO data

Successful modeling of non-stationary signals

Abstract

This paper is concerned with a state-space approach to deep Gaussian process (DGP) regression. We construct the DGP by hierarchically putting transformed Gaussian process (GP) priors on the length scales and magnitudes of the next level of Gaussian processes in the hierarchy. The idea of the state-space approach is to represent the DGP as a non-linear hierarchical system of linear stochastic differential equations (SDEs), where each SDE corresponds to a conditional GP. The DGP regression problem then becomes a state estimation problem, and we can estimate the state efficiently with sequential methods by using the Markov property of the state-space DGP. The computational complexity scales linearly with respect to the number of measurements. Based on this, we formulate state-space MAP as well as Bayesian filtering and smoothing solutions to the DGP regression problem. We demonstrate the performance of the proposed models and methods on synthetic non-stationary signals and apply the state-space DGP to detection of the gravitational waves from LIGO measurements.