Exchangeable Sequence Models Quantify Uncertainty Over Latent Concepts

TL;DR

This paper demonstrates that pre-trained sequence models can naturally perform probabilistic reasoning over exchangeable data, effectively quantifying uncertainty and updating beliefs as new data is observed, aligning with Bayesian principles.

Contribution

It introduces a framework connecting exchangeable sequence modeling with Bayesian inference, showing how pre-trained autoregressive models can explicitly quantify uncertainty over latent environments.

Findings

Sequence prediction loss correlates with uncertainty quality.

Exchangeability can be encoded via data augmentation, regularization, and causal masking.

Pre-trained models perform implicit Bayesian inference over data.

Abstract

Intelligent agents must be able to articulate its own uncertainty. In this work, we show that pre-trained sequence models are naturally capable of probabilistic reasoning over exchangeable data points -- forming informed beliefs and sharpening them as it gathers more information. A sequence model learns the relationship between observations, which differs from typical Bayesian models that quantify uncertainty over latent parameters through priors and likelihoods (e.g., topic models). Despite the apparent difference, we illustrate how exchangeable sequence modeling provides a valid Bayesian model by going back to De Finetti's classical predictive view of probabilistic reasoning: uncertainty comes from data that has not been observed yet, rather than latent parameters. From this perspective, pre-training autoregressive models is equivalent to formulating informed beliefs based on prior observations ("empirical Bayes"), and forward generation is equivalent to simulating instantiations of an environment ("posterior inference"). In particular, exchangeable sequence models can explicitly perform statistical inference; epistemic uncertainty over latent environments is captured by variation in predicted future observations. Formally, we show the sequence prediction loss controls the quality of uncertainty quantification, and propose several approaches for encoding exchangeability in sequence model architectures: data augmentation, regularization, and causal masking.