Exploration and Exploitation of Unlabeled Data for Open-Set Semi-Supervised Learning

TL;DR

This paper introduces a novel open-set semi-supervised learning approach that leverages both in-distribution and out-of-distribution unlabeled data through prototype clustering and importance sampling, improving learning performance.

Contribution

It proposes a prototype-based clustering algorithm and an importance sampling method to effectively utilize both ID and OOD samples in open-set SSL, surpassing previous methods.

Findings

Achieves state-of-the-art results on several benchmarks.

Improves SSL performance even without ID samples in unlabeled data.

Enhances feature learning through prototype-based clustering.

Abstract

In this paper, we address a complex but practical scenario in semi-supervised learning (SSL) named open-set SSL, where unlabeled data contain both in-distribution (ID) and out-of-distribution (OOD) samples. Unlike previous methods that only consider ID samples to be useful and aim to filter out OOD ones completely during training, we argue that the exploration and exploitation of both ID and OOD samples can benefit SSL. To support our claim, i) we propose a prototype-based clustering and identification algorithm that explores the inherent similarity and difference among samples at feature level and effectively cluster them around several predefined ID and OOD prototypes, thereby enhancing feature learning and facilitating ID/OOD identification; ii) we propose an importance-based sampling method that exploits the difference in importance of each ID and OOD sample to SSL, thereby reducing the sampling bias and improving the training. Our proposed method achieves state-of-the-art in several challenging benchmarks, and improves upon existing SSL methods even when ID samples are totally absent in unlabeled data.