Some asymptotic behaviors for the plug-in estimator of entropy

TL;DR

This paper investigates the asymptotic properties of the plug-in estimator for Shannon's entropy on a finite, dynamically varying alphabet, establishing results like asymptotic normality, Berry-Esseen bounds, and moderate deviation principles.

Contribution

It provides new theoretical insights into the asymptotic behaviors of the plug-in entropy estimator with a changing alphabet as sample size grows.

Findings

Asymptotic normality of the estimator

Berry-Esseen bounds established

Moderate deviation principles derived

Abstract

In the present paper, we consider the plug-in estimator of Shannon's entropy defined on a finite alphabet which is assumed to dynamically vary as the sample size increases. The asymptotic behaviors for the plug-in estimator, such as, asymptotic normality, Berry-Esseen bound and moderate deviation principle, are established.