Towards Flexibility and Interpretability of Gaussian Process State-Space Model

TL;DR

This paper introduces TGPSSMs, a flexible and interpretable probabilistic state-space model that uses normalizing flows to enhance Gaussian process priors, improving modeling power and applicability to complex scenarios.

Contribution

We propose TGPSSMs, a novel class of models combining normalizing flows with Gaussian process priors, along with a scalable variational inference algorithm for better flexibility and interpretability.

Findings

TGPSSM outperforms existing methods on synthetic and real datasets.

The normalizing flow enrichment improves model expressivity.

The inference algorithm is efficient and interpretable.

Abstract

The Gaussian process state-space model (GPSSM) has garnered considerable attention over the past decade. However, the standard GP with a preliminary kernel, such as the squared exponential kernel or Mat\'{e}rn kernel, that is commonly used in GPSSM studies, limits the model's representation power and substantially restricts its applicability to complex scenarios. To address this issue, we propose a new class of probabilistic state-space models called TGPSSMs, which leverage a parametric normalizing flow to enrich the GP priors in the standard GPSSM, enabling greater flexibility and expressivity. Additionally, we present a scalable variational inference algorithm that offers a flexible and optimal structure for the variational distribution of latent states. The proposed algorithm is interpretable and computationally efficient due to the sparse GP representation and the bijective nature of normalizing flow. Moreover, we incorporate a constrained optimization framework into the algorithm to enhance the state-space representation capabilities and optimize the hyperparameters, leading to superior learning and inference performance. Experimental results on synthetic and real datasets corroborate that the proposed TGPSSM outperforms several state-of-the-art methods. The accompanying source code is available at \url{https://github.com/zhidilin/TGPSSM}.