A simple derivation and classification of common probability distributions based on information symmetry and measurement scale

TL;DR

This paper unifies common probability distributions into a single framework based on maximum entropy principles and measurement scale, explaining their familial relations through information invariance.

Contribution

It introduces a novel classification of distributions grounded in information symmetry and measurement scale, extending Pearson's phenomenological approach with conceptual foundations.

Findings

Distributions are derived from maximum entropy under specific constraints.

Measurement scale variations explain the familial relations among distributions.

The framework unifies diverse distributions into a coherent system.

Abstract

Commonly observed patterns typically follow a few distinct families of probability distributions. Over one hundred years ago, Karl Pearson provided a systematic derivation and classification of the common continuous distributions. His approach was phenomenological: a differential equation that generated common distributions without any underlying conceptual basis for why common distributions have particular forms and what explains the familial relations. Pearson's system and its descendants remain the most popular systematic classification of probability distributions. Here, we unify the disparate forms of common distributions into a single system based on two meaningful and justifiable propositions. First, distributions follow maximum entropy subject to constraints, where maximum entropy is equivalent to minimum information. Second, different problems associate magnitude to information in different ways, an association we describe in terms of the relation between information invariance and measurement scale. Our framework relates the different continuous probability distributions through the variations in measurement scale that change each family of maximum entropy distributions into a distinct family.