Deep Active Learning in the Open World

TL;DR

This paper introduces ALOE, an active learning algorithm for open-world scenarios that efficiently discovers and learns new classes by combining diversity sampling and energy-based OOD detection, improving model adaptation.

Contribution

The paper presents ALOE, a novel two-stage active learning method that enhances open-world learning by effectively discovering and incorporating new classes with limited annotation budgets.

Findings

ALOE outperforms traditional active learning methods on long-tailed image classification benchmarks.

ALOE accelerates class discovery and improves known-class performance.

A crucial tradeoff exists between known-class accuracy and new class discovery.

Abstract

Machine learning models deployed in open-world scenarios often encounter unfamiliar conditions and perform poorly in unanticipated situations. As AI systems advance and find application in safety-critical domains, effectively handling out-of-distribution (OOD) data is crucial to building open-world learning systems. In this work, we introduce ALOE, a novel active learning algorithm for open-world environments designed to enhance model adaptation by incorporating new OOD classes via a two-stage approach. First, diversity sampling selects a representative set of examples, followed by energy-based OOD detection to prioritize likely unknown classes for annotation. This strategy accelerates class discovery and learning, even under constrained annotation budgets. Evaluations on three long-tailed image classification benchmarks demonstrate that ALOE outperforms traditional active learning baselines, effectively expanding known categories while balancing annotation cost. Our findings reveal a crucial tradeoff between enhancing known-class performance and discovering new classes, setting the stage for future advancements in open-world machine learning.