Exploiting Counter-Examples for Active Learning with Partial labels

TL;DR

This paper introduces a novel active learning approach with partial labels, utilizing counter-examples and a model called WorseNet to improve sample selection and reduce overfitting, demonstrating superior performance across multiple datasets.

Contribution

We propose leveraging counter-examples inspired by human inference to enhance active learning with partial labels, introducing WorseNet to improve sample selection and model robustness.

Findings

WorseNet improves active learning performance across datasets.

Counter-examples help reduce overfitting in partial label learning.

Our method outperforms ten baseline frameworks in experiments.

Abstract

This paper studies a new problem, \emph{active learning with partial labels} (ALPL). In this setting, an oracle annotates the query samples with partial labels, relaxing the oracle from the demanding accurate labeling process. To address ALPL, we first build an intuitive baseline that can be seamlessly incorporated into existing AL frameworks. Though effective, this baseline is still susceptible to the \emph{overfitting}, and falls short of the representative partial-label-based samples during the query process. Drawing inspiration from human inference in cognitive science, where accurate inferences can be explicitly derived from \emph{counter-examples} (CEs), our objective is to leverage this human-like learning pattern to tackle the \emph{overfitting} while enhancing the process of selecting representative samples in ALPL. Specifically, we construct CEs by reversing the partial labels for each instance, and then we propose a simple but effective WorseNet to directly learn from this complementary pattern. By leveraging the distribution gap between WorseNet and the predictor, this adversarial evaluation manner could enhance both the performance of the predictor itself and the sample selection process, allowing the predictor to capture more accurate patterns in the data. Experimental results on five real-world datasets and four benchmark datasets show that our proposed method achieves comprehensive improvements over ten representative AL frameworks, highlighting the superiority of WorseNet. The source code will be available at \url{https://github.com/Ferenas/APLL}.