Bridging Expectation Signals: LLM-Based Experiments and a Behavioral Kalman Filter Framework

TL;DR

This paper investigates how large language models (LLMs) update their expectations using a Behavioral Kalman Filter framework, revealing biases and differences in signal weighting, with implications for economic modeling.

Contribution

It introduces a novel framework to quantify LLM expectation updates and compares behavioral patterns between different agent types, highlighting biases and effects of fine-tuning.

Findings

Agents overweight individual signals over aggregate signals.

Multiple signals reduce the weight of each signal.

LoRA fine-tuning reduces behavioral biases.

Abstract

As LLMs increasingly function as economic agents, the specific mechanisms LLMs use to update their belief with heterogeneous signals remain opaque. We design experiments and develop a Behavioral Kalman Filter framework to quantify how LLM-based agents update expectations, acting as households or firm CEOs, update expectations when presented with individual and aggregate signals. The results from experiments and model estimation reveal four consistent patterns: (1) agents' weighting of priors and signals deviates from unity; (2) both household and firm CEO agents place substantially larger weights on individual signals compared to aggregate signals; (3) we identify a significant and negative interaction between concurrent signals, implying that the presence of multiple information sources diminishes the marginal weight assigned to each individual signal; and (4) expectation formation patterns differ significantly between household and firm CEO agents. Finally, we demonstrate that LoRA fine-tuning mitigates, but does not fully eliminate, behavioral biases in LLM expectation formation.