The dynamics of discovery and the Heaps-Zipf relationship

TL;DR

This paper explores how temporal correlations in sequences like music or web browsing impact the growth of unique elements, challenging the assumption that such growth solely reflects frequency distributions.

Contribution

It demonstrates that temporal structure significantly influences type-token growth, revealing deviations from traditional Zipf-Heaps models in real-world data.

Findings

Temporal correlations cause deviations from Zipf-Heaps predictions.

A minimal model reproduces diverse type-token trajectories.

Type growth depends on both frequency distribution and temporal structure.

Abstract

When following a sequence - such as reading a text or tracking a user's activity - one can measure how the "dictionary" of distinct elements (types) grows with the number of observations (tokens). When this growth follows a power law, it is referred to as Heaps' law, a regularity often associated with Zipf's law and frequently used to characterize human discovery processes. While random sampling from a Zipf-like distribution can reproduce Heaps' law, this connection relies on the assumption of temporal independence - an assumption often violated in real-world systems although frequently found in the literature. Here, we investigate how temporal correlations in token sequences affect the type-token curve. In human behaviors like music listening and web browsing, domain-specific correlations in token ordering lead to systematic deviations from the Zipf-Heaps framework, effectively decoupling the type-token plot from the rank-frequency distribution. Using a minimal one-parameter model, we reproduce a wide variety of type-token trajectories, including the extremal cases that bound all possible behaviors compatible with a given frequency distribution. Our results demonstrate that type-token growth reflects not only the empirical distribution of type frequencies, but also the domain-specific, temporal structure of the sequence - a factor often overlooked in empirical applications of scaling laws to characterize human behavior.