Sum of Ranked Range Loss for Supervised Learning

TL;DR

This paper introduces the sum of ranked range (SoRR) as a versatile framework for forming learning objectives, with applications in classification tasks, and demonstrates its effectiveness through empirical experiments on synthetic and real datasets.

Contribution

The paper proposes the SoRR framework for aggregating values in learning objectives and develops specific loss functions for classification, enhancing robustness to outliers.

Findings

Effective optimization of SoRR achieved with DCA.

AoRR and TKML losses improve robustness in multi-label learning.

Empirical results validate the proposed methods on various datasets.

Abstract

In forming learning objectives, one oftentimes needs to aggregate a set of individual values to a single output. Such cases occur in the aggregate loss, which combines individual losses of a learning model over each training sample, and in the individual loss for multi-label learning, which combines prediction scores over all class labels. In this work, we introduce the sum of ranked range (SoRR) as a general approach to form learning objectives. A ranked range is a consecutive sequence of sorted values of a set of real numbers. The minimization of SoRR is solved with the difference of convex algorithm (DCA). We explore two applications in machine learning of the minimization of the SoRR framework, namely the AoRR aggregate loss for binary/multi-class classification at the sample level and the TKML individual loss for multi-label/multi-class classification at the label level. A combination loss of AoRR and TKML is proposed as a new learning objective for improving the robustness of multi-label learning in the face of outliers in sample and labels alike. Our empirical results highlight the effectiveness of the proposed optimization frameworks and demonstrate the applicability of proposed losses using synthetic and real data sets.