Bayesian Learning from Sequential Data using Gaussian Processes with Signature Covariances

TL;DR

This paper introduces a Bayesian method using Gaussian processes with signature kernels for sequential data, enabling sequence comparison and leveraging stochastic analysis, with scalable sparse variational techniques and integration with neural networks for improved time series classification.

Contribution

It presents a novel Bayesian framework with signature kernels for sequential data, combining GPs with neural networks and introducing scalable sparse variational methods.

Findings

Effective sequence comparison via signature kernels

Scalable sparse variational approach with inducing tensors

Enhanced time series classification performance

Abstract

We develop a Bayesian approach to learning from sequential data by using Gaussian processes (GPs) with so-called signature kernels as covariance functions. This allows to make sequences of different length comparable and to rely on strong theoretical results from stochastic analysis. Signatures capture sequential structure with tensors that can scale unfavourably in sequence length and state space dimension. To deal with this, we introduce a sparse variational approach with inducing tensors. We then combine the resulting GP with LSTMs and GRUs to build larger models that leverage the strengths of each of these approaches and benchmark the resulting GPs on multivariate time series (TS) classification datasets. Code available at https://github.com/tgcsaba/GPSig.