Can OOD Object Detectors Learn from Foundation Models?

TL;DR

This paper explores using large foundation models to generate synthetic out-of-distribution samples, improving OOD object detection performance by augmenting training data with minimal synthetic data.

Contribution

It introduces SyncOOD, a novel data curation method leveraging foundation models to synthesize OOD samples for enhanced detection accuracy.

Findings

SyncOOD outperforms existing methods on multiple benchmarks.

Synthetic OOD data improves ID/OOD decision boundary.

Minimal synthetic data suffices for significant gains.

Abstract

Out-of-distribution (OOD) object detection is a challenging task due to the absence of open-set OOD data. Inspired by recent advancements in text-to-image generative models, such as Stable Diffusion, we study the potential of generative models trained on large-scale open-set data to synthesize OOD samples, thereby enhancing OOD object detection. We introduce SyncOOD, a simple data curation method that capitalizes on the capabilities of large foundation models to automatically extract meaningful OOD data from text-to-image generative models. This offers the model access to open-world knowledge encapsulated within off-the-shelf foundation models. The synthetic OOD samples are then employed to augment the training of a lightweight, plug-and-play OOD detector, thus effectively optimizing the in-distribution (ID)/OOD decision boundaries. Extensive experiments across multiple benchmarks demonstrate that SyncOOD significantly outperforms existing methods, establishing new state-of-the-art performance with minimal synthetic data usage.