Deep Active Learning with Budget Annotation

TL;DR

This paper introduces a hybrid active learning approach that combines uncertainty and informativeness metrics, leveraging pre-trained models to efficiently select and label data within a budget, reducing annotation costs.

Contribution

It proposes a novel hybrid method for active learning that accounts for informativeness and uncertainty, utilizing pre-trained models to minimize labeling costs.

Findings

Effective in reducing annotation costs

Improves data selection accuracy

Demonstrates superior performance across datasets

Abstract

Digital data collected over the decades and data currently being produced with use of information technology is vastly the unlabeled data or data without description. The unlabeled data is relatively easy to acquire but expensive to label even with use of domain experts. Most of the recent works focus on use of active learning with uncertainty metrics measure to address this problem. Although most uncertainty selection strategies are very effective, they fail to take informativeness of the unlabeled instances into account and are prone to querying outliers. In order to address these challenges we propose an hybrid approach of computing both the uncertainty and informativeness of an instance, then automaticaly label the computed instances using budget annotator. To reduce the annotation cost, we employ the state-of-the-art pre-trained models in order to avoid querying information already contained in those models. Our extensive experiments on different sets of datasets demonstrate the efficacy of the proposed approach.