New multicategory boosting algorithms based on multicategory Fisher-consistent losses

TL;DR

This paper extends Fisher-consistent loss functions to multicategory classification, establishing theoretical foundations and deriving new boosting algorithms based on exponential and logistic losses.

Contribution

It introduces the Fisher-consistency condition for multicategory problems and develops novel boosting algorithms using margin-vector-based losses.

Findings

Established Fisher-consistency for multicategory losses

Derived two new multicategory boosting algorithms

Characterized a broad class of smooth convex loss functions

Abstract

Fisher-consistent loss functions play a fundamental role in the construction of successful binary margin-based classifiers. In this paper we establish the Fisher-consistency condition for multicategory classification problems. Our approach uses the margin vector concept which can be regarded as a multicategory generalization of the binary margin. We characterize a wide class of smooth convex loss functions that are Fisher-consistent for multicategory classification. We then consider using the margin-vector-based loss functions to derive multicategory boosting algorithms. In particular, we derive two new multicategory boosting algorithms by using the exponential and logistic regression losses.