The Role of Active Learning in Modern Machine Learning

TL;DR

This paper critically evaluates the effectiveness of Active Learning in modern machine learning, revealing it is less efficient than data augmentation and semi-supervised learning for low data scenarios, but can still enhance performance when combined with these methods.

Contribution

It demonstrates that Active Learning is less effective alone but can improve results when integrated with data augmentation and semi-supervised learning techniques.

Findings

AL generates only 1-4% lift over random sampling.

DA and SSL can achieve up to 60% lift.

AL provides marginal improvements when combined with DA and SSL.

Abstract

Even though Active Learning (AL) is widely studied, it is rarely applied in contexts outside its own scientific literature. We posit that the reason for this is AL's high computational cost coupled with the comparatively small lifts it is typically able to generate in scenarios with few labeled points. In this work we study the impact of different methods to combat this low data scenario, namely data augmentation (DA), semi-supervised learning (SSL) and AL. We find that AL is by far the least efficient method of solving the low data problem, generating a lift of only 1-4\% over random sampling, while DA and SSL methods can generate up to 60\% lift in combination with random sampling. However, when AL is combined with strong DA and SSL techniques, it surprisingly is still able to provide improvements. Based on these results, we frame AL not as a method to combat missing labels, but as the final building block to squeeze the last bits of performance out of data after appropriate DA and SSL methods as been applied.