The common patterns of abundance: the log series and Zipf's law

TL;DR

This paper presents a unified theoretical framework explaining why Zipf's law and the log series are common abundance patterns across various domains, based on invariances and conserved averages.

Contribution

It introduces a general theory that unifies Zipf's law and the log series as endpoints of a broader probability distribution framework.

Findings

Zipf's law arises in language, city sizes, and corporate sizes.

The log series pattern appears in biological species abundances.

The theory encompasses intermediate patterns between Zipf's law and the log series.

Abstract

In a language corpus, the probability that a word occurs $n$ times is often proportional to $1/n^2$. Assigning rank, $s$, to words according to their abundance, $\log s$ vs $\log n$ typically has a slope of minus one. That simple Zipf's law pattern also arises in the population sizes of cities, the sizes of corporations, and other patterns of abundance. By contrast, for the abundances of different biological species, the probability of a population of size $n$ is typically proportional to $1/n$, declining exponentially for larger $n$, the log series pattern. This article shows that the differing patterns of Zipf's law and the log series arise as the opposing endpoints of a more general theory. The general theory follows from the generic form of all probability patterns as a consequence of conserved average values and the associated invariances of scale. To understand the common patterns of abundance, the generic form of probability distributions plus the conserved average abundance is sufficient. The general theory includes cases that are between the Zipf and log series endpoints, providing a broad framework for analyzing widely observed abundance patterns.