Statistical estimation of quadratic R\'enyi entropy for a stationary m-dependent sequence

TL;DR

This paper develops and analyzes U-statistic estimators for quadratic R'enyi entropy of stationary m-dependent sequences, providing their asymptotic properties useful in dependent data scenarios.

Contribution

It introduces new estimators for quadratic R'enyi entropy in dependent sequences and establishes their asymptotic behaviors, extending entropy estimation methods beyond independent data.

Findings

Establishes consistency and asymptotic normality of the estimators.

Derives Poisson convergence results for the estimators.

Applicable to dependent data in time series and distribution identification.

Abstract

The R\'enyi entropy is a generalization of the Shannon entropy and is widely used in mathematical statistics and applied sciences for quantifying the uncertainty in a probability distribution. We consider estimation of the quadratic R\'enyi entropy and related functionals for the marginal distribution of a stationary m-dependent sequence. The U-statistic estimators under study are based on the number of epsilon-close vector observations in the corresponding sample. A variety of asymptotic properties for these estimators are obtained (e.g., consistency, asymptotic normality, Poisson convergence). The results can be used in diverse statistical and computer science problems whenever the conventional independence assumption is too strong (e.g., epsilon-keys in time series databases, distribution identification problems for dependent samples).