Skeptical binary inferences in multi-label problems with sets of probabilities

TL;DR

This paper develops a distributionally robust approach for skeptical inferences in multi-label problems, especially under Hamming loss, demonstrating improved computational efficiency and better handling of uncertain or incomplete data.

Contribution

It introduces a novel framework for skeptical, distributionally robust inferences in multi-label classification, with specific focus on Hamming loss, and provides empirical evidence of its advantages.

Findings

Enhanced computational efficiency with theoretical methods

Produces relevant cautious predictions on hard instances

Handles imperfect information better than existing rejection methods

Abstract

In this paper, we consider the problem of making distributionally robust, skeptical inferences for the multi-label problem, or more generally for Boolean vectors. By distributionally robust, we mean that we consider a set of possible probability distributions, and by skeptical we understand that we consider as valid only those inferences that are true for every distribution within this set. Such inferences will provide partial predictions whenever the considered set is sufficiently big. We study in particular the Hamming loss case, a common loss function in multi-label problems, showing how skeptical inferences can be made in this setting. Our experimental results are organised in three sections; (1) the first one indicates the gain computational obtained from our theoretical results by using synthetical data sets, (2) the second one indicates that our approaches produce relevant cautiousness on those hard-to-predict instances where its precise counterpart fails, and (3) the last one demonstrates experimentally how our approach copes with imperfect information (generated by a downsampling procedure) better than the partial abstention [31] and the rejection rules.