Density estimation with atoms, and functional estimation for mixed discrete-continuous data

TL;DR

This paper introduces a simple modification to existing density estimation methods that enables consistent estimation in mixed discrete-continuous data scenarios without additional assumptions, improving robustness and practical performance.

Contribution

A straightforward wrapper approach that adapts existing density estimators to handle mixed data, removing the need for knowledge of discrete support or mixing proportions.

Findings

Method achieves consistency in mixed data scenarios.

Improved empirical performance over traditional methods.

No additional assumptions required for the modified estimators.

Abstract

In classical density (or density-functional) estimation, it is standard to assume that the underlying distribution has a density with respect to the Lebesgue measure. However, when the data distribution is a mixture of continuous and discrete components, the resulting methods are inconsistent in theory and perform poorly in practice. In this paper, we point out that a minor modification of existing methods for nonparametric density (functional) estimation can allow us to fully remove this assumption while retaining nearly identical theoretical guarantees and improved empirical performance. Our approach is very simple: data points that appear exactly once are likely to originate from the continuous component, whereas repeated observations are indicative of the discrete part. Leveraging this observation, we modify existing estimators for a broad class of functionals of the continuous component of the mixture; this modification is a "wrapper" in the sense that the user can use any underlying method of their choice for continuous density functional estimation. Our modifications deliver consistency without requiring knowledge of the discrete support, the mixing proportion, and without imposing additional assumptions beyond those needed in the absence of the discrete part. Thus, various theorems and existing software packages can be made automatically more robust, with absolutely no additional price when the data is not truly mixed.