Credal Self-Supervised Learning

TL;DR

This paper introduces credal self-supervised learning, allowing models to express uncertainty with sets of probability distributions, improving semi-supervised learning especially when labeled data is scarce.

Contribution

It proposes a novel credal set-based pseudo-labeling approach that better captures uncertainty than traditional probability distributions.

Findings

Outperforms state-of-the-art methods in low-label scenarios

Demonstrates improved uncertainty representation

Shows competitive or superior results in empirical evaluations

Abstract

Self-training is an effective approach to semi-supervised learning. The key idea is to let the learner itself iteratively generate "pseudo-supervision" for unlabeled instances based on its current hypothesis. In combination with consistency regularization, pseudo-labeling has shown promising performance in various domains, for example in computer vision. To account for the hypothetical nature of the pseudo-labels, these are commonly provided in the form of probability distributions. Still, one may argue that even a probability distribution represents an excessive level of informedness, as it suggests that the learner precisely knows the ground-truth conditional probabilities. In our approach, we therefore allow the learner to label instances in the form of credal sets, that is, sets of (candidate) probability distributions. Thanks to this increased expressiveness, the learner is able to represent uncertainty and a lack of knowledge in a more flexible and more faithful manner. To learn from weakly labeled data of that kind, we leverage methods that have recently been proposed in the realm of so-called superset learning. In an exhaustive empirical evaluation, we compare our methodology to state-of-the-art self-supervision approaches, showing competitive to superior performance especially in low-label scenarios incorporating a high degree of uncertainty.