Nearest neighbor density functional estimation from inverse Laplace transform

TL;DR

This paper introduces a novel $L_2$-consistent estimator for general density functionals using $k$-nearest neighbor distances and inverse Laplace transforms, unifying and extending existing entropy and divergence estimators.

Contribution

It proposes a new estimator framework based on inverse Laplace transforms that is asymptotically unbiased and applicable to a broad class of density functionals, including new estimators for various entropies and divergences.

Findings

Estimator is $L_2$-consistent for broad density classes.

Recovers existing estimators for Shannon and Rényi entropies.

Establishes convergence rates for smooth, bounded densities.

Abstract

A new approach to $L_2$-consistent estimation of a general density functional using $k$-nearest neighbor distances is proposed, where the functional under consideration is in the form of the expectation of some function $f$ of the densities at each point. The estimator is designed to be asymptotically unbiased, using the convergence of the normalized volume of a $k$-nearest neighbor ball to a Gamma distribution in the large-sample limit, and naturally involves the inverse Laplace transform of a scaled version of the function $f.$ Some instantiations of the proposed estimator recover existing $k$-nearest neighbor based estimators of Shannon and R\'enyi entropies and Kullback--Leibler and R\'enyi divergences, and discover new consistent estimators for many other functionals such as logarithmic entropies and divergences. The $L_2$-consistency of the proposed estimator is established for a broad class of densities for general functionals, and the convergence rate in mean squared error is established as a function of the sample size for smooth, bounded densities.