Binned semiparametric Bayesian networks for efficient kernel density estimation

TL;DR

This paper presents a novel binned semiparametric Bayesian network model that significantly improves the efficiency of kernel density estimation while maintaining comparable accuracy, addressing computational challenges in high-dimensional data.

Contribution

The paper introduces two new conditional probability distributions for binned semiparametric Bayesian networks that reduce computational costs and mitigate the curse of dimensionality.

Findings

Achieves similar structural learning and log-likelihood estimation as non-binned models.

Offers higher computational speed without sacrificing statistical performance.

Demonstrates effectiveness across synthetic and real datasets with various configurations.

Abstract

This paper introduces a new type of probabilistic semiparametric model that takes advantage of data binning to reduce the computational cost of kernel density estimation in nonparametric distributions. Two new conditional probability distributions are developed for the new binned semiparametric Bayesian networks, the sparse binned kernel density estimation and the Fourier kernel density estimation. These two probability distributions address the curse of dimensionality, which typically impacts binned models, by using sparse tensors and restricting the number of parent nodes in conditional probability calculations. To evaluate the proposal, we perform a complexity analysis and conduct several comparative experiments using synthetic data and datasets from the UCI Machine Learning repository. The experiments include different binning rules, parent restrictions, grid sizes, and number of instances to get a holistic view of the model's behavior. As a result, our binned semiparametric Bayesian networks achieve structural learning and log-likelihood estimations with no statistically significant differences compared to the semiparametric Bayesian networks, but at a much higher speed. Thus, the new binned semiparametric Bayesian networks prove to be a reliable and more efficient alternative to their non-binned counterparts.