On the consistency of Multithreshold Entropy Linear Classifier

TL;DR

This paper analyzes the consistency of the Multithreshold Entropy Linear Classifier (MELC), demonstrating its theoretical connection to margin-based classifiers and validating findings through numerical experiments on multiple datasets.

Contribution

It provides a theoretical analysis of MELC's consistency and its relation to margin-based classifiers, supported by empirical experiments.

Findings

MELC's objective bounds misclassified points similar to hinge loss.

Theoretical link between MELC and maximum margin models.

Numerical experiments confirm theoretical insights.

Abstract

Multithreshold Entropy Linear Classifier (MELC) is a recent classifier idea which employs information theoretic concept in order to create a multithreshold maximum margin model. In this paper we analyze its consistency over multithreshold linear models and show that its objective function upper bounds the amount of misclassified points in a similar manner like hinge loss does in support vector machines. For further confirmation we also conduct some numerical experiments on five datasets.