Prediction stability as a criterion in active learning

TL;DR

This paper introduces a new sequential-based active learning method using prediction stability during training, which is effective on fewer-labeled datasets and compares favorably to traditional methods.

Contribution

It proposes a novel sequential-based active learning approach based on prediction stability, differing from prior methods that only use post-training information.

Findings

Prediction stability is effective on CIFAR-10 and CIFAR-100.

It achieves comparable accuracy to entropy-based methods on CIFAR-10.

It outperforms traditional methods on CIFAR-100.

Abstract

Recent breakthroughs made by deep learning rely heavily on large number of annotated samples. To overcome this shortcoming, active learning is a possible solution. Beside the previous active learning algorithms that only adopted information after training, we propose a new class of method based on the information during training, named sequential-based method. An specific criterion of active learning called prediction stability is proposed to prove the feasibility of sequential-based methods. Experiments are made on CIFAR-10 and CIFAR-100, and the results indicates that prediction stability is effective and works well on fewer-labeled datasets. Prediction stability reaches the accuracy of traditional acquisition functions like entropy on CIFAR-10, and notably outperforms them on CIFAR-100.