Improving Multi-class Classifier Using Likelihood Ratio Estimation with Regularization

TL;DR

This paper enhances the universal-set naive Bayes classifier by integrating a regularized likelihood ratio estimator, effectively addressing imbalanced data issues and improving classification accuracy.

Contribution

It introduces a regularized likelihood ratio estimator into UNB, specifically targeting imbalanced classification problems, which was not considered in prior work.

Findings

Improved classification performance on imbalanced datasets

Effective adjustment of classification scores based on class balance

Regularization suppresses LR overestimation for low-frequency data

Abstract

The universal-set naive Bayes classifier (UNB)~\cite{Komiya:13}, defined using likelihood ratios (LRs), was proposed to address imbalanced classification problems. However, the LR estimator used in the UNB overestimates LRs for low-frequency data, degrading the classification performance. Our previous study~\cite{Kikuchi:19} proposed an effective LR estimator even for low-frequency data. This estimator uses regularization to suppress the overestimation, but we did not consider imbalanced data. In this paper, we integrated the estimator with the UNB. Our experiments with imbalanced data showed that our proposed classifier effectively adjusts the classification scores according to the class balance using regularization parameters and improves the classification performance.