Low-Budget Label Query through Domain Alignment Enforcement

TL;DR

This paper introduces a low-budget label query method that efficiently selects samples for labeling from unlabeled data, leveraging improved domain alignment and prediction consistency to maximize classification accuracy with minimal labeling effort.

Contribution

It presents a novel approach combining enhanced unsupervised domain adaptation with a deterministic sample selection method for low-budget labeling tasks.

Findings

Achieved state-of-the-art results on multiple UDA tasks.

The proposed sample selection method outperforms baselines and competing models.

Effective in maximizing classification accuracy with minimal labeled data.

Abstract

Deep learning revolution happened thanks to the availability of a massive amount of labelled data which have contributed to the development of models with extraordinary inference capabilities. Despite the public availability of a large quantity of datasets, to address specific requirements it is often necessary to generate a new set of labelled data. Quite often, the production of labels is costly and sometimes it requires specific know-how to be fulfilled. In this work, we tackle a new problem named low-budget label query that consists in suggesting to the user a small (low budget) set of samples to be labelled, from a completely unlabelled dataset, with the final goal of maximizing the classification accuracy on that dataset. In this work we first improve an Unsupervised Domain Adaptation (UDA) method to better align source and target domains using consistency constraints, reaching the state of the art on a few UDA tasks. Finally, using the previously trained model as reference, we propose a simple yet effective selection method based on uniform sampling of the prediction consistency distribution, which is deterministic and steadily outperforms other baselines as well as competing models on a large variety of publicly available datasets.