Boosting the Discriminant Power of Naive Bayes

TL;DR

This paper introduces a feature augmentation approach using stacked auto-encoders to enhance naive Bayes classifiers by reducing noise and increasing discriminant power, leading to improved performance on benchmark datasets.

Contribution

It proposes a novel auto-encoder based feature augmentation method that significantly improves naive Bayes classification accuracy by noise reduction and feature expansion.

Findings

Outperforms state-of-the-art naive Bayes classifiers on benchmarks.

Effectively reduces noise and redundant information in features.

Enhances class separation in higher-dimensional feature space.

Abstract

Naive Bayes has been widely used in many applications because of its simplicity and ability in handling both numerical data and categorical data. However, lack of modeling of correlations between features limits its performance. In addition, noise and outliers in the real-world dataset also greatly degrade the classification performance. In this paper, we propose a feature augmentation method employing a stack auto-encoder to reduce the noise in the data and boost the discriminant power of naive Bayes. The proposed stack auto-encoder consists of two auto-encoders for different purposes. The first encoder shrinks the initial features to derive a compact feature representation in order to remove the noise and redundant information. The second encoder boosts the discriminant power of the features by expanding them into a higher-dimensional space so that different classes of samples could be better separated in the higher-dimensional space. By integrating the proposed feature augmentation method with the regularized naive Bayes, the discrimination power of the model is greatly enhanced. The proposed method is evaluated on a set of machine-learning benchmark datasets. The experimental results show that the proposed method significantly and consistently outperforms the state-of-the-art naive Bayes classifiers.