Large Language Models as Discounted Bayesian Filters

TL;DR

This paper introduces a Bayesian filtering framework to analyze how large language models update beliefs over time, revealing they function as discounted Bayesian filters with model-specific forgetting factors, which impacts their reasoning in dynamic environments.

Contribution

It presents a novel probabilistic probe suite and a Bayesian filtering perspective to understand LLM belief updates, highlighting systematic discounting of old evidence and proposing recalibration strategies.

Findings

LLMs' belief updates resemble exponential forgetting filters.

Model-specific discount factors are consistently less than one.

Prompting strategies can recalibrate priors with minimal cost.

Abstract

Large Language Models (LLMs) demonstrate strong few-shot generalization through in-context learning, yet their reasoning in dynamic and stochastic environments remains opaque. Prior studies mainly focus on static tasks and overlook the online adaptation required when beliefs must be continuously updated, which is a key capability for LLMs acting as world models or agents. We introduce a Bayesian filtering framework to evaluate online inference in LLMs. Our probabilistic probe suite spans both multivariate discrete distributions, such as dice rolls, and continuous distributions, such as Gaussian processes, where ground-truth parameters shift over time. We find that while LLM belief updates resemble Bayesian posteriors, they are more accurately characterized by an exponential forgetting filter with a model-specific discount factor smaller than one. This reveals systematic discounting of older evidence that varies significantly across model architectures. Although inherent priors are often miscalibrated, the updating mechanism itself remains structured and principled. We further validate these findings in a simulated agent task and propose prompting strategies that effectively recalibrate priors with minimal computational cost.