How to read probability distributions as statements about process

TL;DR

This paper presents a framework for interpreting probability distributions as expressions of information, revealing how they relate to measurement scales and invariances, thereby enabling a deeper understanding of the processes generating observed patterns.

Contribution

It introduces a novel perspective that reads probability distributions as measurement scales of information, linking invariances to common distribution patterns.

Findings

Most distributions can be understood through information invariance.

Measurement scales are defined by invariance to transformations.

The framework unifies understanding of probability patterns.

Abstract

Probability distributions can be read as simple expressions of information. Each continuous probability distribution describes how information changes with magnitude. Once one learns to read a probability distribution as a measurement scale of information, opportunities arise to understand the processes that generate the commonly observed patterns. Probability expressions may be parsed into four components: the dissipation of all information, except the preservation of average values, taken over the measurement scale that relates changes in observed values to changes in information, and the transformation from the underlying scale on which information dissipates to alternative scales on which probability pattern may be expressed. Information invariances set the commonly observed measurement scales and the relations between them. In particular, a measurement scale for information is defined by its invariance to specific transformations of underlying values into measurable outputs. Essentially all common distributions can be understood within this simple framework of information invariance and measurement scale.