On Initial Pools for Deep Active Learning

TL;DR

This paper investigates whether using intelligently sampled initial labeled pools, especially with self-supervised methods, can enhance deep active learning performance, but results are inconclusive except for promising VAE-based strategies.

Contribution

It explores the impact of self-supervised and unsupervised sampling strategies for initial pools in deep active learning, a less-studied aspect compared to query functions.

Findings

VAE-based initial pool sampling shows promising trends

No conclusive evidence that intelligent sampling outperforms random sampling in the long run

Experimental setup and methodology were peer-reviewed before conducting experiments

Abstract

Active Learning (AL) techniques aim to minimize the training data required to train a model for a given task. Pool-based AL techniques start with a small initial labeled pool and then iteratively pick batches of the most informative samples for labeling. Generally, the initial pool is sampled randomly and labeled to seed the AL iterations. While recent studies have focused on evaluating the robustness of various query functions in AL, little to no attention has been given to the design of the initial labeled pool for deep active learning. Given the recent successes of learning representations in self-supervised/unsupervised ways, we study if an intelligently sampled initial labeled pool can improve deep AL performance. We investigate the effect of intelligently sampled initial labeled pools, including the use of self-supervised and unsupervised strategies, on deep AL methods. The setup, hypotheses, methodology, and implementation details were evaluated by peer review before experiments were conducted. Experimental results could not conclusively prove that intelligently sampled initial pools are better for AL than random initial pools in the long run, although a Variational Autoencoder-based initial pool sampling strategy showed interesting trends that merit deeper investigation.