Building One-class Detector for Anything: Open-vocabulary Zero-shot OOD Detection Using Text-image Models

TL;DR

This paper introduces a zero-shot, open-vocabulary one-class detector leveraging text-image models to identify out-of-distribution inputs across diverse, challenging datasets, enhancing reliability in deep learning applications.

Contribution

It presents a novel one-class open-set OOD detection method using pre-trained text-image models, capable of identifying any out-of-domain data in a zero-shot manner.

Findings

Outperforms previous methods on multiple benchmarks

Effective in detecting fine-grained and semantically similar OOD classes

Handles distribution shifts and multi-object images successfully

Abstract

We focus on the challenge of out-of-distribution (OOD) detection in deep learning models, a crucial aspect in ensuring reliability. Despite considerable effort, the problem remains significantly challenging in deep learning models due to their propensity to output over-confident predictions for OOD inputs. We propose a novel one-class open-set OOD detector that leverages text-image pre-trained models in a zero-shot fashion and incorporates various descriptions of in-domain and OOD. Our approach is designed to detect anything not in-domain and offers the flexibility to detect a wide variety of OOD, defined via fine- or coarse-grained labels, or even in natural language. We evaluate our approach on challenging benchmarks including large-scale datasets containing fine-grained, semantically similar classes, distributionally shifted images, and multi-object images containing a mixture of in-domain and OOD objects. Our method shows superior performance over previous methods on all benchmarks. Code is available at https://github.com/gyhandy/One-Class-Anything