State-space deep Gaussian processes with applications

TL;DR

This paper introduces state-space deep Gaussian processes modeled as stochastic differential equations, enabling efficient Bayesian filtering and smoothing for complex signal modeling and estimation tasks.

Contribution

It presents a novel state-space formulation of deep Gaussian processes using SDEs and develops TME-based filters and smoothers for improved regression and signal estimation.

Findings

Efficient Bayesian filtering and smoothing for deep Gaussian processes.

Accurate estimation of SDE drift functions from partial observations.

Application to spectro-temporal signal feature extraction.

Abstract

This thesis is mainly concerned with state-space approaches for solving deep (temporal) Gaussian process (DGP) regression problems. More specifically, we represent DGPs as hierarchically composed systems of stochastic differential equations (SDEs), and we consequently solve the DGP regression problem by using state-space filtering and smoothing methods. The resulting state-space DGP (SS-DGP) models generate a rich class of priors compatible with modelling a number of irregular signals/functions. Moreover, due to their Markovian structure, SS-DGPs regression problems can be solved efficiently by using Bayesian filtering and smoothing methods. The second contribution of this thesis is that we solve continuous-discrete Gaussian filtering and smoothing problems by using the Taylor moment expansion (TME) method. This induces a class of filters and smoothers that can be asymptotically exact in predicting the mean and covariance of stochastic differential equations (SDEs) solutions. Moreover, the TME method and TME filters and smoothers are compatible with simulating SS-DGPs and solving their regression problems. Lastly, this thesis features a number of applications of state-space (deep) GPs. These applications mainly include, (i) estimation of unknown drift functions of SDEs from partially observed trajectories and (ii) estimation of spectro-temporal features of signals.