Statistical Inference for R\'enyi Entropy Functionals

TL;DR

This paper develops and analyzes estimators for Re9nyi entropy functionals based on sample data, establishing their asymptotic properties for use in statistical and computational applications.

Contribution

It introduces a new class of estimators for Re9nyi entropy functionals using generalized U-statistics and proves their asymptotic behavior.

Findings

Establishes consistency of the estimators.

Proves asymptotic normality of the estimators.

Applicable to problems in computer science and statistics.

Abstract

Numerous entropy-type characteristics (functionals) generalizing R\'enyi entropy are widely used in mathematical statistics, physics, information theory, and signal processing for characterizing uncertainty in probability distributions and distribution identification problems. We consider estimators of some entropy (integral) functionals for discrete and continuous distributions based on the number of epsilon-close vector records in the corresponding independent and identically distributed samples from two distributions. The estimators form a triangular scheme of generalized U-statistics. We show the asymptotic properties of these estimators (e.g., consistency and asymptotic normality). The results can be applied in various problems in computer science and mathematical statistics (e.g., approximate matching for random databases, record linkage, image matching).