Sampling More, Getting Less: Calibration is the Diversity Bottleneck in LLMs

TL;DR

This paper investigates how probability distribution miscalibration during decoding causes diversity collapse in large language models, affecting their output variety and proposing diagnostic tools to analyze this issue.

Contribution

It introduces a validity-diversity framework that decomposes diversity collapse into order and shape miscalibration, providing formal analysis and empirical diagnostics.

Findings

Diversity collapse stems from order and shape miscalibration in LLMs.

Local calibration failures compound over decoding steps, reducing diversity.

Across 14 models, diversity issues are linked to distribution miscalibration, not sampling heuristics.

Abstract

Diversity is essential for language-model applications ranging from creative generation to scientific discovery, yet modern LLMs often collapse into a narrow subset of plausible outputs. While prior work has developed benchmarks for measuring this lack of diversity, less is known about how the step-by-step probability distributions at inference time cause the problem. We introduce a validity--diversity framework that attributes diversity collapse to how an LLM allocates probability mass across valid and invalid continuations during decoding. This framework decomposes the bottleneck into two complementary forms of miscalibration. First, order calibration: valid tokens are not reliably ranked above invalid tokens, so rank-based cutoff rules must trade off between recovering valid continuations and admitting invalid ones. Second, shape calibration: probability mass is overly concentrated only on few valid continuations while having a heavy-tail of mixed valid and invalid tokens, so maintaining high validity limits diversity. We formalize both mechanisms and show that local failures compound across decoding steps, producing strong sequence-level losses in diversity. Empirically, we develop controlled diagnostics for probing these bottlenecks, including tasks with exactly known valid sets and oracle cutoff baselines. Across 14 language models spanning multiple families and scales, we find that diversity collapse is not merely a limitation of particular sampling heuristics, but a consequence of order and shape miscalibration in the LLM distribution.