Improving usual Naive Bayes classifier performances with Neural Naive Bayes based models

TL;DR

This paper proposes Neural Naive Bayes models that enhance traditional Naive Bayes classifiers by incorporating neural networks, significantly improving performance in sentiment analysis tasks.

Contribution

It introduces Neural Naive Bayes and Neural Pooled Markov Chain models to address limitations of traditional Naive Bayes, notably handling complex features and relaxing independence assumptions.

Findings

Error rate reduced by 4.5 times on IMDB dataset

Neural models outperform traditional Naive Bayes in sentiment analysis

Demonstrates effectiveness of neural integration in probabilistic classifiers

Abstract

Naive Bayes is a popular probabilistic model appreciated for its simplicity and interpretability. However, the usual form of the related classifier suffers from two major problems. First, as caring about the observations' law, it cannot consider complex features. Moreover, it considers the conditional independence of the observations given the hidden variable. This paper introduces the original Neural Naive Bayes, modeling the parameters of the classifier induced from the Naive Bayes with neural network functions. This allows to correct the first problem. We also introduce new Neural Pooled Markov Chain models, alleviating the independence condition. We empirically study the benefits of these models for Sentiment Analysis, dividing the error rate of the usual classifier by 4.5 on the IMDB dataset with the FastText embedding.