Testing Parametric Distribution Family Assumptions via Differences in Differential Entropy

TL;DR

The paper proposes a new statistical test, DDE, for determining the underlying parametric distribution of data by comparing entropy estimates, offering a flexible, efficient, and tuning-free approach grounded in information theory.

Contribution

It introduces the DDE test, a unified, asymptotically valid method for testing parametric distribution assumptions using differential entropy comparisons.

Findings

The DDE test is computationally efficient and easy to implement.

It performs well across various distribution families in simulations.

The method requires no tuning parameters or complex regularity conditions.

Abstract

We introduce a broadly applicable statistical procedure for testing which parametric distribution family generated a random sample of data. The method, termed the Difference in Differential Entropy (DDE) test, provides a unified framework applicable to a wide range of distributional families, with asymptotic validity grounded in established maximum likelihood, bootstrap, and kernel density estimation principles. The test is straightforward to implement, computationally efficient, and requires no tuning parameters or specialized regularity conditions. It compares an MLE-based estimate of differential entropy under the null hypothesis with a nonparametric bootstrapped kernel density estimate, using their divergence as an information-theoretic measure of model fit.