Learning with Symmetric Label Noise: The Importance of Being Unhinged

TL;DR

This paper introduces the unhinged loss, a convex and classification-calibrated loss that is robust to symmetric label noise, unlike traditional convex losses, and demonstrates its effectiveness through theoretical analysis and experiments.

Contribution

The paper proposes the unhinged loss, a novel convex loss function that is robust to symmetric label noise and connects to regularized SVM solutions.

Findings

Unhinged loss is SLN-robust and convex.

Strong regularization yields SLN-robust solutions.

Experiments confirm the robustness of the unhinged loss.

Abstract

Convex potential minimisation is the de facto approach to binary classification. However, Long and Servedio [2010] proved that under symmetric label noise (SLN), minimisation of any convex potential over a linear function class can result in classification performance equivalent to random guessing. This ostensibly shows that convex losses are not SLN-robust. In this paper, we propose a convex, classification-calibrated loss and prove that it is SLN-robust. The loss avoids the Long and Servedio [2010] result by virtue of being negatively unbounded. The loss is a modification of the hinge loss, where one does not clamp at zero; hence, we call it the unhinged loss. We show that the optimal unhinged solution is equivalent to that of a strongly regularised SVM, and is the limiting solution for any convex potential; this implies that strong l2 regularisation makes most standard learners SLN-robust. Experiments confirm the SLN-robustness of the unhinged loss.