Out-of-Distribution Detection for Continual Learning: Design Principles and Benchmarking

TL;DR

This paper explores principles and benchmarks for detecting out-of-distribution data in continual learning scenarios, aiming to improve model robustness and adaptability in dynamic real-world environments.

Contribution

It introduces design principles and benchmarking methods specifically for OOD detection within continual learning frameworks, a novel combination in the field.

Findings

Proposes a set of design principles for OOD detection in continual learning.

Develops benchmarking protocols to evaluate OOD detection methods in continual settings.

Demonstrates improved detection performance using proposed methods on benchmark datasets.

Abstract

Recent years have witnessed significant progress in the development of machine learning models across a wide range of fields, fueled by increased computational resources, large-scale datasets, and the rise of deep learning architectures. From malware detection to enabling autonomous navigation, modern machine learning systems have demonstrated remarkable capabilities. However, as these models are deployed in ever-changing real-world scenarios, their ability to remain reliable and adaptive over time becomes increasingly important. For example, in the real world, new malware families are continuously developed, whereas autonomous driving cars are employed in many different cities and weather conditions. Models trained in fixed settings can not respond effectively to novel conditions encountered post-deployment. In fact, most machine learning models are still developed under the assumption that training and test data are independent and identically distributed (i.i.d.), i.e., sampled from the same underlying (unknown) distribution. While this assumption simplifies model development and evaluation, it does not hold in many real-world applications, where data changes over time and unexpected inputs frequently occur. Retraining models from scratch whenever new data appears is computationally expensive, time-consuming, and impractical in resource-constrained environments. These limitations underscore the need for Continual Learning (CL), which enables models to incrementally learn from evolving data streams without forgetting past knowledge, and Out-of-Distribution (OOD) detection, which allows systems to identify and respond to novel or anomalous inputs. Jointly addressing both challenges is critical to developing robust, efficient, and adaptive AI systems.