A Benchmark Study of Classical and Dual Polynomial Regression (DPR)-Based Probability Density Estimation Technique

TL;DR

This paper introduces a computationally efficient Dual Polynomial Regression (DPR) method for probability density estimation, leveraging piecewise modeling and GPU-accelerated kernel density estimation to better capture asymmetries in real-world data.

Contribution

The paper proposes a novel DPR approach that improves density estimation accuracy and efficiency by combining piecewise polynomial fitting with GPU-accelerated KDE and HDE methods.

Findings

DPR with tKDE achieves high accuracy in modeling asymmetric unimodal distributions.

GPU-based KDE and HDE significantly outperform traditional Python implementations.

DPR order 4 balances accuracy and computational efficiency on real-world data.

Abstract

The probability density function (PDF) plays a central role in statistical and machine learning modeling. Real-world data often deviates from Gaussian assumptions, exhibiting skewness and exponential decay. To evaluate how well different density estimation methods capture such irregularities, we generated six unimodal datasets from diverse distributions that reflect real-world anomalies. These were compared using parametric methods (Pearson Type I and Normal distribution) as well as non-parametric approaches, including histograms, kernel density estimation (KDE), and our proposed method. To accelerate computation, we implemented GPU-based versions of KDE (tKDE) and histogram estimation (tHDE) in TensorFlow, both of which outperform Python SciPy's KDE. Prior work demonstrated the use of piecewise modeling for density estimation, such as local polynomial regression; however, these methods are computationally intensive. Based on the concept of piecewise modeling, we developed a computationally efficient model, the Dual Polynomial Regression (DPR) method, which leverages tKDE or tHDE for training. DPR employs the piecewise strategy to split the PDF at its mode and fit polynomial regressions to the left and right halves independently, enabling better capture of the asymmetric shape of the unimodal distribution. We used the Mean Squared Error (MSE), Jensen-Shannon Divergence (JSD), and Pearson's correlation coefficient, with reference to the baseline PDF, to validate accuracy. We verified normalization using Area Under the Curve (AUC) and computational overhead via execution time. Validation on real-world systolic and diastolic data from 300,000 unique patients shows that the DPR of order 4, trained with tKDE, offers the best balance between accuracy and computational overhead.