A Scalable Algorithm for Active Learning

TL;DR

This paper introduces a scalable, approximate active learning algorithm for multiclass classification that maintains accuracy while significantly reducing computational and storage costs, and demonstrates its effectiveness on large datasets with GPU acceleration.

Contribution

It proposes an efficient approximate algorithm for active learning that scales better than FIRAL and includes a GPU parallel implementation, with maintained accuracy on large datasets.

Findings

Achieves comparable accuracy to FIRAL on benchmark datasets.

Demonstrates strong and weak scaling on up to 12 GPUs with three million points.

Reduces storage and computational complexity significantly.

Abstract

FIRAL is a recently proposed deterministic active learning algorithm for multiclass classification using logistic regression. It was shown to outperform the state-of-the-art in terms of accuracy and robustness and comes with theoretical performance guarantees. However, its scalability suffers when dealing with datasets featuring a large number of points $n$, dimensions $d$, and classes $c$, due to its $\mathcal{O}(c^2d^2+nc^2d)$ storage and $\mathcal{O}(c^3(nd^2 + bd^3 + bn))$ computational complexity where $b$ is the number of points to select in active learning. To address these challenges, we propose an approximate algorithm with storage requirements reduced to $\mathcal{O}(n(d+c) + cd^2)$ and a computational complexity of $\mathcal{O}(bncd^2)$. Additionally, we present a parallel implementation on GPUs. We demonstrate the accuracy and scalability of our approach using MNIST, CIFAR-10, Caltech101, and ImageNet. The accuracy tests reveal no deterioration in accuracy compared to FIRAL. We report strong and weak scaling tests on up to 12 GPUs, for three million point synthetic dataset.