Recursive Learning of Asymptotic Variational Objectives

TL;DR

This paper introduces OSIWAE, an online variational inference method for state-space models that uses a recursive approach with sequential Monte Carlo, enabling real-time learning of model parameters and latent states.

Contribution

The paper proposes a novel online variational inference framework for SSMs that maximizes an asymptotic variational lower bound using stochastic approximation and SMC, improving theoretical grounding and efficiency.

Findings

Efficient online learning of model parameters demonstrated.

The method outperforms existing online variational SMC approaches.

Theoretical analysis confirms the validity of the variational objective.

Abstract

General state-space models (SSMs) are widely used in statistical machine learning and are among the most classical generative models for sequential time-series data. SSMs, comprising latent Markovian states, can be subjected to variational inference (VI), but standard VI methods like the importance-weighted autoencoder (IWAE) lack functionality for streaming data. To enable online VI in SSMs when the observations are received in real time, we propose maximising an IWAE-type variational lower bound on the asymptotic contrast function, rather than the standard IWAE ELBO, using stochastic approximation. Unlike the recursive maximum likelihood method, which directly maximises the asymptotic contrast, our approach, called online sequential IWAE (OSIWAE), allows for online learning of both model parameters and a Markovian recognition model for inferring latent states. By approximating filter state posteriors and their derivatives using sequential Monte Carlo (SMC) methods, we create a particle-based framework for online VI in SSMs. This approach is more theoretically well-founded than recently proposed online variational SMC methods. We provide rigorous theoretical results on the learning objective and a numerical study demonstrating the method's efficiency in learning model parameters and particle proposal kernels.