Model-Centric and Data-Centric Aspects of Active Learning for Deep Neural Networks

TL;DR

This paper provides a comprehensive analysis of active learning for deep neural networks, examining training modes, model configurations, query strategies, statistical behaviors, and the use of pseudo-labels to improve learning efficiency.

Contribution

It offers a unified study of various aspects of active learning with deep neural networks, proposing more efficient query strategies and insights into their behaviors.

Findings

Incremental and cumulative training modes impact learning efficiency.

Optimized query strategies improve informativeness and reduce labeling effort.

Pseudo-labels can enhance active learning performance.

Abstract

We study different aspects of active learning with deep neural networks in a consistent and unified way. i) We investigate incremental and cumulative training modes which specify how the newly labeled data are used for training. ii) We study active learning w.r.t. the model configurations such as the number of epochs and neurons as well as the choice of batch size. iii) We consider in detail the behavior of query strategies and their corresponding informativeness measures and accordingly propose more efficient querying procedures. iv) We perform statistical analyses, e.g., on actively learned classes and test error estimation, that reveal several insights about active learning. v) We investigate how active learning with neural networks can benefit from pseudo-labels as proxies for actual labels.