Consistent second-order discrete kernel smoothing using dispersed Conway-Maxwell-Poisson kernels

TL;DR

This paper introduces a novel second-order discrete kernel smoothing method based on Conway-Maxwell-Poisson kernels, improving probability mass function estimation for discrete data with flexible dispersion handling.

Contribution

The paper develops a new second-order discrete kernel smoother using Conway-Maxwell-Poisson kernels and proposes two automated bandwidth selection methods, enhancing estimation accuracy and flexibility.

Findings

Excellent performance in small and large samples

Outperforms existing estimators in simulations

Effective in real-world biological data modeling

Abstract

The histogram estimator of a discrete probability mass function often exhibits undesirable properties related to zero probability estimation both within the observed range of counts and outside into the tails of the distribution. To circumvent this, we formulate a novel second-order discrete kernel smoother based on the recently developed mean-parametrized Conway--Maxwell--Poisson distribution which allows for both over- and under-dispersion. Two automated bandwidth selection approaches, one based on a simple minimization of the Kullback--Leibler divergence and another based on a more computationally demanding cross-validation criterion, are introduced. Both methods exhibit excellent small- and large-sample performance. Computational results on simulated datasets from a range of target distributions illustrate the flexibility and accuracy of the proposed method compared to existing smoothed and unsmoothed estimators. The method is applied to the modelling of somite counts in earthworms, and the number of development days of insect pests on the Hura tree.