A Symmetric Loss Perspective of Reliable Machine Learning

TL;DR

This paper reviews symmetric surrogate losses in binary classification, highlighting their robustness to label corruption and applications in NLP, while discussing future directions for reliable machine learning.

Contribution

It provides a comprehensive overview of symmetric losses, their robustness benefits, and explores their applications and potential future research directions.

Findings

Symmetric losses improve robustness in corrupted label scenarios.

Application of robust AUC maximization in NLP tasks.

Discussion of future research directions in symmetric loss design.

Abstract

When minimizing the empirical risk in binary classification, it is a common practice to replace the zero-one loss with a surrogate loss to make the learning objective feasible to optimize. Examples of well-known surrogate losses for binary classification include the logistic loss, hinge loss, and sigmoid loss. It is known that the choice of a surrogate loss can highly influence the performance of the trained classifier and therefore it should be carefully chosen. Recently, surrogate losses that satisfy a certain symmetric condition (aka., symmetric losses) have demonstrated their usefulness in learning from corrupted labels. In this article, we provide an overview of symmetric losses and their applications. First, we review how a symmetric loss can yield robust classification from corrupted labels in balanced error rate (BER) minimization and area under the receiver operating characteristic curve (AUC) maximization. Then, we demonstrate how the robust AUC maximization method can benefit natural language processing in the problem where we want to learn only from relevant keywords and unlabeled documents. Finally, we conclude this article by discussing future directions, including potential applications of symmetric losses for reliable machine learning and the design of non-symmetric losses that can benefit from the symmetric condition.