Combining pre-trained Vision Transformers and CIDER for Out Of Domain Detection

TL;DR

This paper evaluates the effectiveness of pre-trained Vision Transformers and CNNs for out-of-domain detection, demonstrating that combining these models with CIDER refinement enhances detection performance and establishes a new baseline.

Contribution

It introduces the use of pre-trained Vision Transformers for OOD detection and shows that combining them with CIDER improves accuracy beyond existing methods.

Findings

Pre-trained transformers outperform traditional models in OOD detection.

Combining transformers with CIDER further enhances detection performance.

Transformers set a new baseline for OOD detection in industrial applications.

Abstract

Out-of-domain (OOD) detection is a crucial component in industrial applications as it helps identify when a model encounters inputs that are outside the training distribution. Most industrial pipelines rely on pre-trained models for downstream tasks such as CNN or Vision Transformers. This paper investigates the performance of those models on the task of out-of-domain detection. Our experiments demonstrate that pre-trained transformers models achieve higher detection performance out of the box. Furthermore, we show that pre-trained ViT and CNNs can be combined with refinement methods such as CIDER to improve their OOD detection performance even more. Our results suggest that transformers are a promising approach for OOD detection and set a stronger baseline for this task in many contexts