A Comparative Survey of Deep Active Learning

TL;DR

This paper provides a comprehensive survey and benchmarking of 19 deep active learning methods, offering insights into their performance and factors influencing efficacy to guide future research and applications.

Contribution

It introduces DeepAL+, a toolkit re-implementing 19 DAL methods, and offers a fair comparison framework along with analysis of factors affecting DAL performance.

Findings

Benchmarking of 19 DAL methods across datasets

Identification of key factors influencing DAL efficacy

Guidelines for designing DAL experiments

Abstract

While deep learning (DL) is data-hungry and usually relies on extensive labeled data to deliver good performance, Active Learning (AL) reduces labeling costs by selecting a small proportion of samples from unlabeled data for labeling and training. Therefore, Deep Active Learning (DAL) has risen as a feasible solution for maximizing model performance under a limited labeling cost/budget in recent years. Although abundant methods of DAL have been developed and various literature reviews conducted, the performance evaluation of DAL methods under fair comparison settings is not yet available. Our work intends to fill this gap. In this work, We construct a DAL toolkit, DeepAL+, by re-implementing 19 highly-cited DAL methods. We survey and categorize DAL-related works and construct comparative experiments across frequently used datasets and DAL algorithms. Additionally, we explore some factors (e.g., batch size, number of epochs in the training process) that influence the efficacy of DAL, which provides better references for researchers to design their DAL experiments or carry out DAL-related applications.