Variational Gaussian Process Diffusion Processes

TL;DR

This paper introduces a novel variational inference method for diffusion process models, using a site-based exponential family parameterization to enable faster and more effective learning algorithms.

Contribution

It proposes a new exponential family parameterization of Gaussian variational processes, improving inference speed and learning quality over previous methods.

Findings

The new parameterization enables convex optimization for inference.

The approach improves inference speed compared to fixed-point iteration methods.

It provides a better objective for learning model parameters.

Abstract

Diffusion processes are a class of stochastic differential equations (SDEs) providing a rich family of expressive models that arise naturally in dynamic modelling tasks. Probabilistic inference and learning under generative models with latent processes endowed with a non-linear diffusion process prior are intractable problems. We build upon work within variational inference, approximating the posterior process as a linear diffusion process, and point out pathologies in the approach. We propose an alternative parameterization of the Gaussian variational process using a site-based exponential family description. This allows us to trade a slow inference algorithm with fixed-point iterations for a fast algorithm for convex optimization akin to natural gradient descent, which also provides a better objective for learning model parameters.