Sparse Algorithms for Markovian Gaussian Processes

TL;DR

This paper introduces scalable sparse algorithms for Markovian Gaussian processes that efficiently handle large time series and spatio-temporal data through a site-based approximate inference framework.

Contribution

It develops a general site-based approach for approximate inference in sparse Markovian Gaussian processes, extending existing algorithms with novel sparse methods.

Findings

Algorithms scale linearly with the number of inducing points

Effective for large time series and spatio-temporal data

Applicable to variational inference, expectation propagation, and Kalman smoothers

Abstract

Approximate Bayesian inference methods that scale to very large datasets are crucial in leveraging probabilistic models for real-world time series. Sparse Markovian Gaussian processes combine the use of inducing variables with efficient Kalman filter-like recursions, resulting in algorithms whose computational and memory requirements scale linearly in the number of inducing points, whilst also enabling parallel parameter updates and stochastic optimisation. Under this paradigm, we derive a general site-based approach to approximate inference, whereby we approximate the non-Gaussian likelihood with local Gaussian terms, called sites. Our approach results in a suite of novel sparse extensions to algorithms from both the machine learning and signal processing literature, including variational inference, expectation propagation, and the classical nonlinear Kalman smoothers. The derived methods are suited to large time series, and we also demonstrate their applicability to spatio-temporal data, where the model has separate inducing points in both time and space.