Binary classification with ambiguous training data

TL;DR

This paper introduces a novel binary classification method that effectively incorporates ambiguous samples during training by extending the reject option framework and using a convex upper bound for computational feasibility.

Contribution

It proposes a new approach to train classifiers with ambiguous data by extending the reject option with a loss function that accounts for ambiguity, improving learning from such data.

Findings

The method successfully utilizes ambiguous samples during training.

Numerical experiments show improved classification performance.

The approach is computationally feasible using convex upper bounds.

Abstract

In supervised learning, we often face with ambiguous (A) samples that are difficult to label even by domain experts. In this paper, we consider a binary classification problem in the presence of such A samples. This problem is substantially different from semi-supervised learning since unlabeled samples are not necessarily difficult samples. Also, it is different from 3-class classification with the positive (P), negative (N), and A classes since we do not want to classify test samples into the A class. Our proposed method extends binary classification with reject option, which trains a classifier and a rejector simultaneously using P and N samples based on the 0-1-$c$ loss with rejection cost $c$. More specifically, we propose to train a classifier and a rejector under the 0-1-$c$-$d$ loss using P, N, and A samples, where $d$ is the misclassification penalty for ambiguous samples. In our practical implementation, we use a convex upper bound of the 0-1-$c$-$d$ loss for computational tractability. Numerical experiments demonstrate that our method can successfully utilize the additional information brought by such A training data.