How to Achieve Prototypical Birth and Death for OOD Detection?

TL;DR

This paper introduces PID, a biologically inspired dynamic prototype adjustment method for OOD detection that adaptively varies the number of prototypes during training, leading to improved detection performance.

Contribution

The paper proposes a novel adaptive prototype learning method, PID, which dynamically adjusts prototypes via birth and death mechanisms based on data complexity, enhancing OOD detection.

Findings

PID outperforms existing methods on CIFAR-100 benchmark.

Achieves state-of-the-art FPR95 performance.

Produces more compact and well-separated ID embeddings.

Abstract

Out-of-Distribution (OOD) detection is crucial for the secure deployment of machine learning models, and prototype-based learning methods are among the mainstream strategies for achieving OOD detection. Existing prototype-based learning methods generally rely on a fixed number of prototypes. This static assumption fails to adapt to the inherent complexity differences across various categories. Currently, there is still a lack of a mechanism that can adaptively adjust the number of prototypes based on data complexity. Inspired by the processes of cell birth and death in biology, we propose a novel method named PID (Prototype bIrth and Death) to adaptively adjust the prototype count based on data complexity. This method relies on two dynamic mechanisms during the training process: prototype birth and prototype death. The birth mechanism instantiates new prototypes in data regions with insufficient representation by identifying the overload level of existing prototypes, thereby meticulously capturing intra-class substructures. Conversely, the death mechanism reinforces the decision boundary by pruning prototypes with ambiguous class boundaries through evaluating their discriminability. Through birth and death, the number of prototypes can be dynamically adjusted according to the data complexity, leading to the learning of more compact and better-separated In-Distribution (ID) embeddings, which significantly enhances the capability to detect OOD samples. Experiments demonstrate that our dynamic method, PID, significantly outperforms existing methods on benchmarks such as CIFAR-100, achieving State-of-the-Art (SOTA) performance, especially on the FPR95 metric.