Predicted and Verified Deviations from Zipf's law in Ecology of Competing Products

TL;DR

This paper empirically tests a theory explaining deviations from Zipf's law in product market shares, showing that stochastic growth and birth-death processes account for observed power-law variations without adjustable parameters.

Contribution

It provides the first complete empirical validation of a theory linking stochastic growth and birth-death processes to power-law exponents in complex systems.

Findings

Observed power-law exponents vary with product characteristics.

The theory accurately predicts the variations without adjustable parameters.

Results are applicable to other systems with similar statistical properties.

Abstract

Zipf's power-law distribution is a generic empirical statistical regularity found in many complex systems. However, rather than universality with a single power-law exponent (equal to 1 for Zipf's law), there are many reported deviations that remain unexplained. A recently developed theory finds that the interplay between (i) one of the most universal ingredients, namely stochastic proportional growth, and (ii) birth and death processes, leads to a generic power-law distribution with an exponent that depends on the characteristics of each ingredient. Here, we report the first complete empirical test of the theory and its application, based on the empirical analysis of the dynamics of market shares in the product market. We estimate directly the average growth rate of market shares and its standard deviation, the birth rates and the "death" (hazard) rate of products. We find that temporal variations and product differences of the observed power-law exponents can be fully captured by the theory with no adjustable parameters. Our results can be generalized to many systems for which the statistical properties revealed by power law exponents are directly linked to the underlying generating mechanism.