Efficient Multiclass Implementations of L1-Regularized Maximum Entropy

TL;DR

This paper introduces a new multiclass classification methodology using L1-regularized maximum entropy models, enhancing the SL1-Max algorithm to handle various distribution assumptions and outperform some existing classifiers.

Contribution

It proposes a modified SL1-Max algorithm for multiclass problems and explores diverse modeling assumptions, providing a flexible framework that surpasses traditional methods like AdaBoost and SVMs.

Findings

Modified SL1-Max algorithm effectively handles multiclass categorization.

Various distribution assumptions improve model flexibility.

Maximum Entropy matches or exceeds state-of-the-art classifiers.

Abstract

This paper discusses the application of L1-regularized maximum entropy modeling or SL1-Max [9] to multiclass categorization problems. A new modification to the SL1-Max fast sequential learning algorithm is proposed to handle conditional distributions. Furthermore, unlike most previous studies, the present research goes beyond a single type of conditional distribution. It describes and compares a variety of modeling assumptions about the class distribution (independent or exclusive) and various types of joint or conditional distributions. It results in a new methodology for combining binary regularized classifiers to achieve multiclass categorization. In this context, Maximum Entropy can be considered as a generic and efficient regularized classification tool that matches or outperforms the state-of-the art represented by AdaBoost and SVMs.