Generalized Coverage for More Robust Low-Budget Active Learning

TL;DR

This paper introduces MaxHerding, a robust and efficient generalized coverage method for low-budget active learning that outperforms existing approaches in image classification tasks.

Contribution

It proposes a generalized coverage framework and a new greedy algorithm, MaxHerding, which improves robustness and performance over ProbCover in low-budget active learning.

Findings

MaxHerding outperforms existing active learning methods on multiple benchmarks.

The method is more robust to hyper-parameter choices than ProbCover.

MaxHerding requires less computational cost than most competitive methods.

Abstract

The ProbCover method of Yehuda et al. is a well-motivated algorithm for active learning in low-budget regimes, which attempts to "cover" the data distribution with balls of a given radius at selected data points. We demonstrate, however, that the performance of this algorithm is extremely sensitive to the choice of this radius hyper-parameter, and that tuning it is quite difficult, with the original heuristic frequently failing. We thus introduce (and theoretically motivate) a generalized notion of "coverage," including ProbCover's objective as a special case, but also allowing smoother notions that are far more robust to hyper-parameter choice. We propose an efficient greedy method to optimize this coverage, generalizing ProbCover's algorithm; due to its close connection to kernel herding, we call it "MaxHerding." The objective can also be optimized non-greedily through a variant of $k$-medoids, clarifying the relationship to other low-budget active learning methods. In comprehensive experiments, MaxHerding surpasses existing active learning methods across multiple low-budget image classification benchmarks, and does so with less computational cost than most competitive methods.