Rapid parametric density estimation

TL;DR

This paper introduces inexpensive parametric density estimation methods using polynomial and Fourier series fitting, enabling efficient density reconstruction, distortion correction, and clustering with error decreasing as 1/√n.

Contribution

It proposes novel, computationally efficient density estimation techniques based on averaging monomials or trigonometric functions over samples, extending applications to distortion correction and clustering.

Findings

Error decreases as 1/√n with sample size

Method enables density reconstruction and distortion correction

Applicable to clustering with negative or complex weights

Abstract

Parametric density estimation, for example as Gaussian distribution, is the base of the field of statistics. Machine learning requires inexpensive estimation of much more complex densities, and the basic approach is relatively costly maximum likelihood estimation (MLE). There will be discussed inexpensive density estimation, for example literally fitting a polynomial (or Fourier series) to the sample, which coefficients are calculated by just averaging monomials (or sine/cosine) over the sample. Another discussed basic application is fitting distortion to some standard distribution like Gaussian - analogously to ICA, but additionally allowing to reconstruct the disturbed density. Finally, by using weighted average, it can be also applied for estimation of non-probabilistic densities, like modelling mass distribution, or for various clustering problems by using negative (or complex) weights: fitting a function which sign (or argument) determines clusters. The estimated parameters are approaching the optimal values with error dropping like $1/\sqrt{n}$, where $n$ is the sample size.