Bandwidth Selectors on Semiparametric Bayesian Networks

TL;DR

This paper investigates advanced bandwidth selection methods for kernel density estimators within semiparametric Bayesian networks, demonstrating that cross-validation techniques outperform traditional normal rule approaches, especially with large datasets.

Contribution

It introduces and evaluates state-of-the-art bandwidth selectors for SPBNs, extending an open-source package and providing empirical evidence of their effectiveness over the normal rule.

Findings

Cross-validation outperforms the normal rule in large sample scenarios.

Bandwidth selectors improve density estimation and predictive performance.

Normal rule stagnates with increasing data, while advanced selectors leverage more information.

Abstract

Semiparametric Bayesian networks (SPBNs) integrate parametric and non-parametric probabilistic models, offering flexibility in learning complex data distributions from samples. In particular, kernel density estimators (KDEs) are employed for the non-parametric component. Under the assumption of data normality, the normal rule is used to learn the bandwidth matrix for the KDEs in SPBNs. This matrix is the key hyperparameter that controls the trade-off between bias and variance. However, real-world data often deviates from normality, potentially leading to suboptimal density estimation and reduced predictive performance. This paper first establishes the theoretical framework for the application of state-of-the-art bandwidth selectors and subsequently evaluates their impact on SPBN performance. We explore the approaches of cross-validation and plug-in selectors, assessing their effectiveness in enhancing the learning capability and applicability of SPBNs. To support this investigation, we have extended the open-source package PyBNesian for SPBNs with the additional bandwidth selection techniques and conducted extensive experimental analyses. Our results demonstrate that the proposed bandwidth selectors leverage increasing information more effectively than the normal rule, which, despite its robustness, stagnates with more data. In particular, unbiased cross-validation generally outperforms the normal rule, highlighting its advantage in high sample size scenarios.