Indirect Maximum Entropy Bandwidth

TL;DR

This paper introduces a novel bandwidth selection method for kernel density and distribution estimation, based on maximizing the entropy of leave-one-out estimates, with practical recommendations from simulation results.

Contribution

It proposes an indirect entropy maximization approach for bandwidth selection, connecting entropy, likelihood, and goodness-of-fit tests in kernel estimation.

Findings

The Anderson-Darling based bandwidth performs reliably across various distributions.

The proposed methods are effective in simulation studies.

The approach is useful for cross-validation and density forecast evaluation.

Abstract

This paper proposes a new method of bandwidth selection in kernel estimation of density and distribution functions motivated by the connection between maximisation of the entropy of probability integral transforms and maximum likelihood in classical parametric models. The proposed estimators are designed to indirectly maximise the entropy of the leave-one-out kernel estimates of a distribution function, which are the analogues of the parametric probability integral transforms. The estimators based on minimisation of the Cramer-von Mises discrepancy, near-solution of the moment-based estimating equations, and inversion of the Neyman smooth test statistic are discussed and their performance compared in a simulation study. The bandwidth minimising the Anderson-Darling statistic is found to perform reliably for a variety of distribution shapes and can be recommended in practice. The results will also be of interest to anyone analysing the cross-validation bandwidths based on leave-one-out estimates or evaluation of nonparametric density forecasts.