Do Language Models Have Bayesian Brains? Distinguishing Stochastic and Deterministic Decision Patterns within Large Language Models

TL;DR

This paper investigates whether large language models make decisions based on probabilistic sampling or deterministic processes, revealing that they can exhibit near-deterministic behavior which challenges previous assumptions about their Bayesian nature.

Contribution

The study introduces a method to distinguish stochastic from deterministic decision patterns in language models, revealing their potential for deterministic decision-making and addressing prior inference issues.

Findings

Language models can produce near-deterministic decisions even with non-zero temperature

Deterministic behavior can lead to false priors during Gibbs sampling

Proposed approach effectively identifies decision patterns in various models

Abstract

Language models are essentially probability distributions over token sequences. Auto-regressive models generate sentences by iteratively computing and sampling from the distribution of the next token. This iterative sampling introduces stochasticity, leading to the assumption that language models make probabilistic decisions, similar to sampling from unknown distributions. Building on this assumption, prior research has used simulated Gibbs sampling, inspired by experiments designed to elicit human priors, to infer the priors of language models. In this paper, we revisit a critical question: Do language models possess Bayesian brains? Our findings show that under certain conditions, language models can exhibit near-deterministic decision-making, such as producing maximum likelihood estimations, even with a non-zero sampling temperature. This challenges the sampling assumption and undermines previous methods for eliciting human-like priors. Furthermore, we demonstrate that without proper scrutiny, a system with deterministic behavior undergoing simulated Gibbs sampling can converge to a "false prior." To address this, we propose a straightforward approach to distinguish between stochastic and deterministic decision patterns in Gibbs sampling, helping to prevent the inference of misleading language model priors. We experiment on a variety of large language models to identify their decision patterns under various circumstances. Our results provide key insights in understanding decision making of large language models.