Assessing uncertainty in Gaussian mixtures-based entropy estimation

TL;DR

This paper introduces a new method for quantifying uncertainty in Gaussian mixture-based entropy estimation using a weighted likelihood bootstrap approach, improving accuracy and empirical coverage.

Contribution

It proposes a novel weighted likelihood bootstrap technique with Dirichlet weights for better uncertainty quantification in mixture-based entropy estimation.

Findings

Weighted bootstrap improves uncertainty estimates.

Dirichlet weights enhance empirical coverage.

Method applied to financial and sports data.

Abstract

Entropy estimation plays a crucial role in various fields, such as information theory, statistical data science, and machine learning. However, traditional entropy estimation methods often struggle with complex data distributions. Mixture-based estimation of entropy has been recently proposed and gained attention due to its ease of use and accuracy. This paper presents a novel approach to quantify the uncertainty associated with this mixture-based entropy estimation method using weighted likelihood bootstrap. Unlike standard methods, our approach leverages the underlying mixture structure by assigning random weights to observations in a weighted likelihood bootstrap procedure, leading to more accurate uncertainty estimation. The generation of weights is also investigated, leading to the proposal of using weights obtained from a specific Dirichlet distribution which, in conjunction with centered percentile intervals, yields the optimal setting to ensure empirical coverage closer to the nominal level. Extensive simulation studies comparing different resampling strategies are presented and results discussed. The proposed approach is illustrated by analyzing the log-returns of daily Gold prices at COMEX for the years 2014--2022, and the Net Rating scores, an advanced statistic used in basketball analytics, for NBA teams with reference to the 2022/23 regular season.