On the Effect of Uncertainty on Layer-wise Inference Dynamics

TL;DR

This paper investigates how uncertainty influences the internal inference process of large language models, revealing that uncertainty does not significantly alter layer-wise probability dynamics, challenging simple uncertainty detection methods.

Contribution

The study introduces the use of the Tuned Lens to analyze layer-wise probability trajectories, showing that uncertainty does not affect inference dynamics as previously assumed.

Findings

Uncertain and certain predictions have similar probability trajectories across layers.

Abrupt confidence increases occur at similar layers for both certain and uncertain outputs.

More competent models may process uncertainty differently, indicating variability in inference dynamics.

Abstract

Understanding how large language models (LLMs) internally represent and process their predictions is central to detecting uncertainty and preventing hallucinations. While several studies have shown that models encode uncertainty in their hidden states, it is underexplored how this affects the way they process such hidden states. In this work, we demonstrate that the dynamics of output token probabilities across layers for certain and uncertain outputs are largely aligned, revealing that uncertainty does not seem to affect inference dynamics. Specifically, we use the Tuned Lens, a variant of the Logit Lens, to analyze the layer-wise probability trajectories of final prediction tokens across 11 datasets and 5 models. Using incorrect predictions as those with higher epistemic uncertainty, our results show aligned trajectories for certain and uncertain predictions that both observe abrupt increases in confidence at similar layers. We balance this finding by showing evidence that more competent models may learn to process uncertainty differently. Our findings challenge the feasibility of leveraging simplistic methods for detecting uncertainty at inference. More broadly, our work demonstrates how interpretability methods may be used to investigate the way uncertainty affects inference.