Analysis of an information-theoretic model for communication

TL;DR

This paper analyzes an information-theoretic model of communication to understand cost minimization and its relation to Zipf's law, revealing that the model does not naturally produce Zipf's law except under specific conditions.

Contribution

The study provides a direct analysis of the cost-minimization problem in a communication model, clarifies the nature of solutions at the critical point, and evaluates the effectiveness of existing algorithms.

Findings

Minimum cost is $ ext{min}( ext{lambda}, 1- ext{lambda})$

Zipf's law appears only at a specific transition point $ ext{lambda}=1/2$

Existing algorithms correctly identify global minima at the transition

Abstract

We study the cost-minimization problem posed by Ferrer i Cancho and Sol\'e in their model of communication that aimed at explaining the origin of Zipf's law [PNAS 100, 788 (2003)]. Direct analysis shows that the minimum cost is $\min {\lambda, 1-\lambda}$, where $\lambda$ determines the relative weights of speaker's and hearer's costs in the total, as shown in several previous works using different approaches. The nature and multiplicity of the minimizing solution changes discontinuously at $\lambda=1/2$, being qualitatively different for $\lambda < 1/2$, $\lambda > 1/2$, and $\lambda=1/2$. Zipf's law is found only in a vanishing fraction of the minimum-cost solutions at $\lambda = 1/2$ and therefore is not explained by this model. Imposing the further condition of equal costs yields distributions substantially closer to Zipf's law, but significant differences persist. We also investigate the solutions reached by the previously used minimization algorithm and find that they correctly recover global minimum states at the transition.