Exact Learning Augmented Naive Bayes Classifier

TL;DR

This paper introduces an exact learning augmented naive Bayes classifier that improves classification accuracy, especially with small sample sizes and complex class variables, by combining exact learning with naive Bayes structure.

Contribution

It proposes a novel exact learning method for naive Bayes classifiers that guarantees asymptotic equivalence to Bayesian network learning, enhancing performance in small data scenarios.

Findings

Exact learning with ML outperforms approximate methods on large data.

Proposed ANB method outperforms existing approaches on small datasets.

ANB achieves asymptotic equivalence to learned Bayesian networks.

Abstract

Earlier studies have shown that classification accuracies of Bayesian networks (BNs) obtained by maximizing the conditional log likelihood (CLL) of a class variable, given the feature variables, were higher than those obtained by maximizing the marginal likelihood (ML). However, differences between the performances of the two scores in the earlier studies may be attributed to the fact that they used approximate learning algorithms, not exact ones. This paper compares the classification accuracies of BNs with approximate learning using CLL to those with exact learning using ML. The results demonstrate that the classification accuracies of BNs obtained by maximizing the ML are higher than those obtained by maximizing the CLL for large data. However, the results also demonstrate that the classification accuracies of exact learning BNs using the ML are much worse than those of other methods when the sample size is small and the class variable has numerous parents. To resolve the problem, we propose an exact learning augmented naive Bayes classifier (ANB), which ensures a class variable with no parents. The proposed method is guaranteed to asymptotically estimate the identical class posterior to that of the exactly learned BN. Comparison experiments demonstrated the superior performance of the proposed method.