Maximal Diversity and Zipf's Law

TL;DR

This paper demonstrates that Zipf's law naturally arises from the maximization of diversity in component sizes within complex systems, linking it to Heaps' law and validating findings with linguistic data.

Contribution

It provides an analytical model showing the co-occurrence of Zipf's law and maximal diversity, connecting these phenomena with Heaps' law and empirical linguistic data.

Findings

Zipf's law emerges from diversity maximization

Derived relationship between diversity and system size

Model aligns well with linguistic datasets

Abstract

Zipf's law describes the empirical size distribution of the components of many systems in natural and social sciences and humanities. We show, by solving a statistical model, that Zipf's law co-occurs with the maximization of the diversity of the component sizes. The law ruling the increase of such diversity with the total dimension of the system is derived and its relation with Heaps' law is discussed. As an example, we show that our analytical results compare very well with linguistics datasets.