ODOV: Towards Open-Domain Open-Vocabulary Object Detection

TL;DR

This paper introduces the ODOV problem, a new benchmark OD-LVIS dataset, and a novel method leveraging language models for open-vocabulary object detection adaptable to real-world domain shifts.

Contribution

It presents the first benchmark for open-domain, open-vocabulary object detection and a novel domain-adaptive detection method using language models.

Findings

The proposed method outperforms existing approaches on the OD-LVIS benchmark.

The benchmark covers diverse real-world domains and categories.

The method effectively adapts to domain and category shifts during testing.

Abstract

In this work, we handle a new problem of Open-Domain Open-Vocabulary (ODOV) object detection, which considers the detection model's adaptability to the real world including both domain and category shifts. For this problem, we first construct a new benchmark OD-LVIS, which includes 46,949 images, covers 18 complex real-world domains and 1,203 categories, and provides a comprehensive dataset for evaluating real-world object detection. Besides, we develop a novel baseline method for ODOV detection.The proposed method first leverages large language models to generate the domain-agnostic text prompts for category embedding. It further learns the domain embedding from the given image, which, during testing, can be integrated into the category embedding to form the customized domain-specific category embedding for each test image. We provide sufficient benchmark evaluations for the proposed ODOV detection task and report the results, which verify the rationale of ODOV detection, the usefulness of our benchmark, and the superiority of the proposed method.