Mind the Gap when Conditioning Amortised Inference in Sequential Latent-Variable Models

TL;DR

This paper reveals that partially-conditioned amortised inference in sequential latent-variable models can lead to suboptimal learning, and demonstrates that fully-conditioned posteriors improve model performance across various tasks.

Contribution

The paper identifies a mismatch caused by the ELBO in partially-conditioned inference and proposes fully-conditioned posteriors to enhance model accuracy.

Findings

Fully-conditioned posteriors outperform partially-conditioned ones.

Theoretical analysis shows the ELBO induces product of smoothing posteriors.

Empirical results demonstrate improved generative modeling and prediction.

Abstract

Amortised inference enables scalable learning of sequential latent-variable models (LVMs) with the evidence lower bound (ELBO). In this setting, variational posteriors are often only partially conditioned. While the true posteriors depend, e.g., on the entire sequence of observations, approximate posteriors are only informed by past observations. This mimics the Bayesian filter -- a mixture of smoothing posteriors. Yet, we show that the ELBO objective forces partially-conditioned amortised posteriors to approximate products of smoothing posteriors instead. Consequently, the learned generative model is compromised. We demonstrate these theoretical findings in three scenarios: traffic flow, handwritten digits, and aerial vehicle dynamics. Using fully-conditioned approximate posteriors, performance improves in terms of generative modelling and multi-step prediction.