A Cross-Conformal Predictor for Multi-label Classification

TL;DR

This paper introduces a novel application of Conformal Prediction to multi-label classification, providing reliable confidence measures for predicted class subsets, which enhances decision-making under high uncertainty.

Contribution

It extends Conformal Prediction to multi-label learning, offering a method that quantifies confidence for each predicted subset of classes.

Findings

Provides confidence measures for multi-label predictions

Enhances reliability of multi-label classifiers

Addresses high uncertainty in multi-label tasks

Abstract

Unlike the typical classification setting where each instance is associated with a single class, in multi-label learning each instance is associated with multiple classes simultaneously. Therefore the learning task in this setting is to predict the subset of classes to which each instance belongs. This work examines the application of a recently developed framework called Conformal Prediction (CP) to the multi-label learning setting. CP complements the predictions of machine learning algorithms with reliable measures of confidence. As a result the proposed approach instead of just predicting the most likely subset of classes for a new unseen instance, also indicates the likelihood of each predicted subset being correct. This additional information is especially valuable in the multi-label setting where the overall uncertainty is extremely high.