Deep Active Learning with a Neural Architecture Search

TL;DR

This paper introduces a novel active learning approach for deep neural networks that dynamically searches for effective architectures during the learning process, significantly improving performance over fixed-architecture methods.

Contribution

It proposes a new active learning strategy that integrates neural architecture search into the active learning process, addressing the limitation of fixed architectures.

Findings

Outperforms fixed-architecture active learning methods

Effective with multiple querying techniques

Demonstrates significant improvements in learning efficiency

Abstract

We consider active learning of deep neural networks. Most active learning works in this context have focused on studying effective querying mechanisms and assumed that an appropriate network architecture is a priori known for the problem at hand. We challenge this assumption and propose a novel active strategy whereby the learning algorithm searches for effective architectures on the fly, while actively learning. We apply our strategy using three known querying techniques (softmax response, MC-dropout, and coresets) and show that the proposed approach overwhelmingly outperforms active learning using fixed architectures.