Estimation of differential entropy for normal populations under prior information

TL;DR

This paper develops improved estimators for the differential entropy of two normal populations considering prior information, providing theoretical derivations, numerical comparisons, and real-world application insights.

Contribution

It introduces new estimators that incorporate prior order restrictions, outperforming existing affine equivariant estimators in entropy estimation.

Findings

Proposed estimators dominate the best affine equivariant estimator (BAEE).

Numerical results show improved risk performance under quadratic and linex loss functions.

Interval estimation methods achieve better coverage probabilities and shorter lengths.

Abstract

The problem of nonlinear functional of parameters, such as differential entropy, has received much attention in information theory and statistics. In many situations, prior information about the parameters is available in the form of order restrictions. This information should be taken into account to obtain improved estimators. In this paper, we study the problems of point-wise and interval estimation of the entropy of two normal populations under a general location-invariant loss function. For the point-wise estimation, we have derived the maximum likelihood estimator (MLE), restricted MLE and the uniformly minimum variance unbiased estimator (UMVUE). Further, we derive a sufficient condition for improvement over affine equivariant estimators. A class of improved estimators is derived that dominates the best affine equivariant estimator (BAEE). Furthermore, we obtain a class of smooth improved estimator that dominates BAEE. We present special loss functions and derive expressions for the proposed improved estimators. A numerical study is conducted to compare the risk performance of the proposed estimators under quadratic and linex loss functions. For interval estimation, we have derived asymptotic confidence interval, bootstrap confidence intervals, HPD credible interval, and intervals based on generalized pivot variables. A comprehensive numerical comparison of these intervals is carried out in terms of coverage probabilities and average lengths. Finally, the proposed results are illustrated with a real example: the failure of the air-conditioning systems on Boeing 720 jet planes.