Zipf and Heaps laws from dependency structures in component systems

TL;DR

This paper introduces a simple, analytically tractable model based on dependency structures that explains the emergence of Zipf and Heaps laws in complex systems with multiple component copies.

Contribution

It extends previous binary models to include component multiplicities and provides a mean-field analytical approach to derive key regularities.

Findings

Reproduces Zipf's law with a core set of components

Shows Heaps law with three regimes: linear, sub-linear, and saturation

Analytical results match numerical simulations

Abstract

Complex natural and technological systems can be considered, on a coarse-grained level, as assemblies of elementary components: for example, genomes as sets of genes, or texts as sets of words. On one hand, the joint occurrence of components emerges from architectural and specific constraints in such systems. On the other hand, general regularities may unify different systems, such as the broadly studied Zipf and Heaps laws, respectively concerning the distribution of component frequencies and their number as a function of system size. Dependency structures (i.e., directed networks encoding the dependency relations between the components in a system) were proposed recently as a possible organizing principles underlying some of the regularities observed. However, the consequences of this assumption were explored only in binary component systems, where solely the presence or absence of components is considered, and multiple copies of the same component are not allowed. Here, we consider a simple model that generates, from a given ensemble of dependency structures, a statistical ensemble of sets of components, allowing for components to appear with any multiplicity. Our model is a minimal extension that is memoryless, and therefore accessible to analytical calculations. A mean-field analytical approach (analogous to the "Zipfian ensemble" in the linguistics literature) captures the relevant laws describing the component statistics as we show by comparison with numerical computations. In particular, we recover a power-law Zipf rank plot, with a set of core components, and a Heaps law displaying three consecutive regimes (linear, sub-linear and saturating) that we characterize quantitatively.