Quadratic Term Correction on Heaps' Law

TL;DR

This paper demonstrates that quadratic functions in log-log scale better fit Heaps' law data, revealing a slight concavity in the word-type versus word-token relationship across various texts.

Contribution

It introduces a quadratic correction to Heaps' law, providing a more accurate model for the type-token relationship in language data.

Findings

Quadratic functions fit the type-token data perfectly.

Regression coefficients indicate a slight concavity in the log-log scale.

A pseudo-variance explains the curvature observed in the data.

Abstract

Heaps' or Herdan's law characterizes the word-type vs. word-token relation by a power-law function, which is concave in linear-linear scale but a straight line in log-log scale. However, it has been observed that even in log-log scale, the type-token curve is still slightly concave, invalidating the power-law relation. At the next-order approximation, we have shown, by twenty English novels or writings (some are translated from another language to English), that quadratic functions in log-log scale fit the type-token data perfectly. Regression analyses of log(type)-log(token) data with both a linear and quadratic term consistently lead to a linear coefficient of slightly larger than 1, and a quadratic coefficient around -0.02. Using the ``random drawing colored ball from the bag with replacement" model, we have shown that the curvature of the log-log scale is identical to a ``pseudo-variance" which is negative. Although a pseudo-variance calculation may encounter numeric instability when the number of tokens is large, due to the large values of pseudo-weights, this formalism provides a rough estimation of the curvature when the number of tokens is small.