Exclusivity Classes and Partitions of Loss Functions

TL;DR

This paper introduces a comprehensive framework for understanding incompatibilities among loss functions through exclusivity regions, classes, and partitions, revealing structural properties and illustrating with examples like quantile and robust regression losses.

Contribution

It develops a novel theoretical framework for classifying loss functions based on optimality incompatibilities, including structural properties and formal realizations.

Findings

Established structural properties of exclusivity objects

Illustrated partitions with quantile, classification, and robust losses

Proposed an open conjecture on minimax exclusivity

Abstract

Loss functions determine what it means for an estimator to be optimal, yet the ways in which different losses impose structurally incompatible optimality requirements are not captured by existing decision-theoretic frameworks. This paper develops a general theory of such incompatibilities by introducing \emph{exclusivity regions}, \emph{exclusivity classes}, and \emph{exclusivity partitions} of the loss space relative to an abstract optimality operator $\mathcal{O}$. An exclusivity region is a subset of losses such that no single estimator can be $\mathcal{O}$-optimal for a loss inside the region and a loss outside it; exclusivity classes additionally require realizability by at least one optimal estimator, and exclusivity partitions provide a global decomposition of a loss family into disjoint exclusivity regions (or classes, if the partition is realizable). We establish basic structural properties of these objects, including the role of conic geometry and invariance of optimality under positive scaling, which allows partitions on normalized representatives to extend along rays in loss cones. The framework is illustrated through three fully formal, nontrivial realizable exclusivity partitions under Bayes risk optimality: asymmetric linear (quantile) losses, convex margin-based classification losses and Huber-type robust regression losses. We also formulate an open conjecture on minimax exclusivity for power-type losses and discuss connections to elicitation theory and to topological regularity properties of losses.