Estimating Continuous Distributions in Bayesian Classifiers

TL;DR

This paper explores nonparametric kernel density estimation for continuous variables in Bayesian classifiers, demonstrating significant error reductions over Gaussian assumptions across various datasets.

Contribution

It introduces the use of kernel density estimation in Bayesian classifiers, moving beyond normality assumptions for improved modeling of continuous variables.

Findings

Kernel density estimation reduces classification error.

Nonparametric methods outperform Gaussian assumptions.

Improved accuracy on natural and artificial datasets.

Abstract

When modeling a probability distribution with a Bayesian network, we are faced with the problem of how to handle continuous variables. Most previous work has either solved the problem by discretizing, or assumed that the data are generated by a single Gaussian. In this paper we abandon the normality assumption and instead use statistical methods for nonparametric density estimation. For a naive Bayesian classifier, we present experimental results on a variety of natural and artificial domains, comparing two methods of density estimation: assuming normality and modeling each conditional distribution with a single Gaussian; and using nonparametric kernel density estimation. We observe large reductions in error on several natural and artificial data sets, which suggests that kernel estimation is a useful tool for learning Bayesian models.