COSTARR: Consolidated Open Set Technique with Attenuation for Robust Recognition

TL;DR

COSTARR introduces a novel open-set recognition method that leverages feature attenuation to better differentiate known from unknown classes, demonstrating significant improvements over existing approaches across multiple datasets and architectures.

Contribution

The paper proposes COSTARR, a new open-set recognition technique that utilizes both familiar features and attenuation-based discarded information, with a probabilistic interpretation and extensive validation.

Findings

COSTARR outperforms prior state-of-the-art methods on multiple datasets.

Both pre- and post-attenuated features are crucial for improved recognition.

The approach generalizes well across various architectures and datasets.

Abstract

Handling novelty remains a key challenge in visual recognition systems. Existing open-set recognition (OSR) methods rely on the familiarity hypothesis, detecting novelty by the absence of familiar features. We propose a novel attenuation hypothesis: small weights learned during training attenuate features and serve a dual role-differentiating known classes while discarding information useful for distinguishing known from unknown classes. To leverage this overlooked information, we present COSTARR, a novel approach that combines both the requirement of familiar features and the lack of unfamiliar ones. We provide a probabilistic interpretation of the COSTARR score, linking it to the likelihood of correct classification and belonging in a known class. To determine the individual contributions of the pre- and post-attenuated features to COSTARR's performance, we conduct ablation studies that show both pre-attenuated deep features and the underutilized post-attenuated Hadamard product features are essential for improving OSR. Also, we evaluate COSTARR in a large-scale setting using ImageNet2012-1K as known data and NINCO, iNaturalist, OpenImage-O, and other datasets as unknowns, across multiple modern pre-trained architectures (ViTs, ConvNeXts, and ResNet). The experiments demonstrate that COSTARR generalizes effectively across various architectures and significantly outperforms prior state-of-the-art methods by incorporating previously discarded attenuation information, advancing open-set recognition capabilities.