Solution for OOD-CV Workshop SSB Challenge 2024 (Open-Set Recognition Track)

TL;DR

This paper presents a hybrid method combining post-hoc OOD detection and test-time augmentation for open-set recognition, achieving competitive results in the ECCV 2024 SSB Challenge.

Contribution

It introduces a fusion of OOD detection techniques and TTA strategies, evaluated across multiple models, to improve open-set recognition performance.

Findings

Achieved AUROC of 79.77, ranking 5th in the challenge.

FPR95 score of 61.44, ranking 2nd in the challenge.

Demonstrated the effectiveness of combining TTA with post-hoc OOD methods.

Abstract

This report provides a detailed description of the method we explored and proposed in the OSR Challenge at the OOD-CV Workshop during ECCV 2024. The challenge required identifying whether a test sample belonged to the semantic classes of a classifier's training set, a task known as open-set recognition (OSR). Using the Semantic Shift Benchmark (SSB) for evaluation, we focused on ImageNet1k as the in-distribution (ID) dataset and a subset of ImageNet21k as the out-of-distribution (OOD) dataset.To address this, we proposed a hybrid approach, experimenting with the fusion of various post-hoc OOD detection techniques and different Test-Time Augmentation (TTA) strategies. Additionally, we evaluated the impact of several base models on the final performance. Our best-performing method combined Test-Time Augmentation with the post-hoc OOD techniques, achieving a strong balance between AUROC and FPR95 scores. Our approach resulted in AUROC: 79.77 (ranked 5th) and FPR95: 61.44 (ranked 2nd), securing second place in the overall competition.