Learning with Symmetric Label Noise: The Importance of Being Unhinged
Brendan van Rooyen, Aditya Krishna Menon, Robert C. Williamson

TL;DR
This paper introduces the unhinged loss, a convex and classification-calibrated loss that is robust to symmetric label noise, unlike traditional convex losses, and demonstrates its effectiveness through theoretical analysis and experiments.
Contribution
The paper proposes the unhinged loss, a novel convex loss function that is robust to symmetric label noise and connects to regularized SVM solutions.
Findings
Unhinged loss is SLN-robust and convex.
Strong regularization yields SLN-robust solutions.
Experiments confirm the robustness of the unhinged loss.
Abstract
Convex potential minimisation is the de facto approach to binary classification. However, Long and Servedio [2010] proved that under symmetric label noise (SLN), minimisation of any convex potential over a linear function class can result in classification performance equivalent to random guessing. This ostensibly shows that convex losses are not SLN-robust. In this paper, we propose a convex, classification-calibrated loss and prove that it is SLN-robust. The loss avoids the Long and Servedio [2010] result by virtue of being negatively unbounded. The loss is a modification of the hinge loss, where one does not clamp at zero; hence, we call it the unhinged loss. We show that the optimal unhinged solution is equivalent to that of a strongly regularised SVM, and is the limiting solution for any convex potential; this implies that strong l2 regularisation makes most standard learners…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsMachine Learning and Data Classification · Machine Learning and Algorithms · Imbalanced Data Classification Techniques
