Descriptive versus Regulatory Uncertainty in Bounded Predictive Systems

TL;DR

This paper distinguishes between descriptive and regulatory uncertainty in predictive systems, proving that current transformer architectures only exhibit descriptive uncertainty and exploring the thermodynamic implications of this limitation.

Contribution

It introduces a formal distinction between types of uncertainty, proves current transformers are confined to descriptive uncertainty, and empirically demonstrates this in language models.

Findings

Transformer models exhibit invariant entropy across tasks despite accuracy variation.

Entropy and accuracy are orthogonal and scale-invariant in models.

Thermodynamic analysis links regulatory uncertainty to physical energy costs.

Abstract

Any system that models the world under finite representational capacity must compress; any compression entails a prior; and the prior is the system's bias. What has not been established is whether uncertainty participates in the dynamics governing future behavior, or merely describes the output distribution without consequence. We introduce a structural distinction between descriptive uncertainty, which does not recursively modulate the system's policy, and regulatory uncertainty, which directly enters the optimization landscape and drives persistent adaptive restructuring. We prove formally that current transformer architectures are confined to descriptive uncertainty at inference. We ground this in thermodynamics via Landauer's principle: for uncertainty to be regulatory, epistemic error must cost real energy; in a decoupled system, hallucinations and correct derivations dissipate identical energy. We test this empirically across three locally-deployed language models (3B, 8B, 70B parameters). Token-level Shannon entropy is statistically invariant across tasks spanning pattern retrieval, causal operator application, and out-of-distribution causal generalization in all three models (all pairwise p >= 0.568; within-model ranges 0.011-0.028 nats), while task accuracy varies substantially across the same conditions (0%-100%). Entropy and accuracy are orthogonal. The decoupling is scale-invariant: larger models achieve higher accuracy but identical entropy flatness. This structural incapacity is not resolvable by additional parameters or training data. Genuine epistemic grounding requires physical coupling between thermodynamic substrate state and information processing cost.