Open-Vocabulary Object Detection with Meta Prompt Representation and Instance Contrastive Optimization

TL;DR

This paper introduces MIC, a novel framework for open-vocabulary object detection that enhances generalization to novel classes using meta prompts and contrastive learning, outperforming previous methods without extra data or complex training.

Contribution

The paper proposes a new MIC framework with meta prompts and contrastive learning to improve open-vocabulary detection and generalization to unseen classes.

Findings

MIC outperforms previous SOTA methods on LVIS, COCO, and Objects365 datasets.

The approach improves AP by +4.3% on COCO and +1.9% on Objects365.

MIC does not require extra data or knowledge distillation during training.

Abstract

Classical object detectors are incapable of detecting novel class objects that are not encountered before. Regarding this issue, Open-Vocabulary Object Detection (OVOD) is proposed, which aims to detect the objects in the candidate class list. However, current OVOD models are suffering from overfitting on the base classes, heavily relying on the large-scale extra data, and complex training process. To overcome these issues, we propose a novel framework with Meta prompt and Instance Contrastive learning (MIC) schemes. Firstly, we simulate a novel-class-emerging scenario to help the prompt learner that learns class and background prompts generalize to novel classes. Secondly, we design an instance-level contrastive strategy to promote intra-class compactness and inter-class separation, which benefits generalization of the detector to novel class objects. Without using knowledge distillation, ensemble model or extra training data during detector training, our proposed MIC outperforms previous SOTA methods trained with these complex techniques on LVIS. Most importantly, MIC shows great generalization ability on novel classes, e.g., with $+4.3\%$ and $+1.9\% \ \mathrm{AP}$ improvement compared with previous SOTA on COCO and Objects365, respectively.