Nearest-Class Mean and Logits Agreement for Wildlife Open-Set Recognition

TL;DR

This paper introduces a post-processing open-set recognition method for wildlife classification that measures agreement between feature-based and logit-based models, achieving high AUROC scores without retraining the classifier.

Contribution

The study proposes a novel post-processing approach using Nearest Class Mean and logits agreement, avoiding retraining and demonstrating consistent performance across datasets.

Findings

Achieves AUROC of 93.41 and 95.35 on two wildlife datasets.

Ranks within the top three on evaluated datasets.

Does not require retraining the pre-trained model.

Abstract

Current state-of-the-art Wildlife classification models are trained under the closed world setting. When exposed to unknown classes, they remain overconfident in their predictions. Open-set Recognition (OSR) aims to classify known classes while rejecting unknown samples. Several OSR methods have been proposed to model the closed-set distribution by observing the feature, logit, or softmax probability space. A significant drawback of many existing approaches is the requirement to retrain the pre-trained classification model with the OSR-specific strategy. This study contributes a post-processing OSR method that measures the agreement between the models' features and predicted logits. We propose a probability distribution based on an input's distance to its Nearest Class Mean (NCM). The NCM-based distribution is then compared with the softmax probabilities from the logit space to measure agreement between the NCM and the classification head. Our proposed strategy ranks within the top three on two evaluated datasets, showing consistent performance across the two datasets. In contrast, current state-of-the-art methods excel on a single dataset. We achieve an AUROC of 93.41 and 95.35 for African and Swedish animals. The code can be found https://github.com/Applied-Representation-Learning-Lab/OSR.