How much is too much? Measuring divergence from Benford's Law with the Equivalent Contamination Proportion (ECP)

TL;DR

This paper introduces the Equivalent Contamination Proportion (ECP), a new metric for measuring divergence from Benford's Law that overcomes limitations of traditional statistics, providing a more interpretable and robust assessment of data conformity.

Contribution

The paper proposes the ECP as a novel, interpretable, and robust measure for divergence from Benford's Law, addressing key limitations of existing statistical tools.

Findings

ECP provides a continuous measure of deviation from Benford's Law.

ECP is robust to sample size and comparable across different divergence statistics.

Application to influential studies demonstrates ECP's utility in data analysis.

Abstract

Conformity with Benford's Law is widely used to detect irregularities in numerical datasets, particularly in accounting, finance, and economics. However, the statistical tools commonly used for this purpose (such as Chi-squared, MAD, or KS) suffer from three key limitations: sensitivity to sample size, lack of interpretability of their scale, and the absence of a common metric that allows for comparison across different statistics. This paper introduces the Equivalent Contamination Proportion (ECP) to address these issues. Defined as the proportion of contamination in a hypothetical Benford-conforming sample such that the expected value of the divergence statistic matches the one observed in the actual data, the ECP provides a continuous and interpretable measure of deviation (ranging from 0 to 1), is robust to sample size, and offers consistent results across different divergence statistics under mild conditions. Closed-form and simulation-based methods are developed for estimating the ECP, and, through a retrospective analysis of three influential studies, it is shown how the ECP can complement the information provided by traditional divergence statistics and enhance the interpretation of results.