The Convexity and Design of Composite Multiclass Losses

TL;DR

This paper investigates the convexity properties of composite multiclass loss functions, providing conditions for their convexity and demonstrating how this framework enables the design of loss functions with consistent Bayes risk.

Contribution

It establishes conditions for the convexity of composite multiclass losses and introduces a framework for designing loss functions with desired properties.

Findings

Derived conditions for strong convexity of composite losses

Showed how to design loss functions with the same Bayes risk

Explored implications of composite loss structure on multiclass prediction

Abstract

We consider composite loss functions for multiclass prediction comprising a proper (i.e., Fisher-consistent) loss over probability distributions and an inverse link function. We establish conditions for their (strong) convexity and explore the implications. We also show how the separation of concerns afforded by using this composite representation allows for the design of families of losses with the same Bayes risk.