Accurate and Diverse Sampling of Sequences based on a "Best of Many" Sample Objective

TL;DR

This paper introduces a 'Best of Many' sample objective for Gaussian Latent Variable models, significantly improving the accuracy and diversity of sequence predictions in uncertain, real-world scenarios.

Contribution

It proposes a novel 'Best of Many' objective that enhances the diversity and accuracy of sequence predictions in latent variable models.

Findings

Outperforms prior models on traffic scene prediction

Achieves better diversity in weather data forecasting

Improves sequence prediction accuracy across tasks

Abstract

For autonomous agents to successfully operate in the real world, anticipation of future events and states of their environment is a key competence. This problem has been formalized as a sequence extrapolation problem, where a number of observations are used to predict the sequence into the future. Real-world scenarios demand a model of uncertainty of such predictions, as predictions become increasingly uncertain -- in particular on long time horizons. While impressive results have been shown on point estimates, scenarios that induce multi-modal distributions over future sequences remain challenging. Our work addresses these challenges in a Gaussian Latent Variable model for sequence prediction. Our core contribution is a "Best of Many" sample objective that leads to more accurate and more diverse predictions that better capture the true variations in real-world sequence data. Beyond our analysis of improved model fit, our models also empirically outperform prior work on three diverse tasks ranging from traffic scenes to weather data.