Restoring Negative Information in Few-Shot Object Detection

TL;DR

This paper introduces a novel metric learning framework that restores negative information in few-shot object detection, significantly improving detection performance by leveraging negative and positive representatives during training and inference.

Contribution

It proposes a new negative- and positive-representative based metric learning framework and inference scheme for few-shot object detection, enhancing the use of negative information.

Findings

Substantial performance improvements on ImageNet-LOC and PASCAL VOC datasets.

Effective encoding of negative information improves detection accuracy.

Outperforms existing state-of-the-art few-shot object detection methods.

Abstract

Few-shot learning has recently emerged as a new challenge in the deep learning field: unlike conventional methods that train the deep neural networks (DNNs) with a large number of labeled data, it asks for the generalization of DNNs on new classes with few annotated samples. Recent advances in few-shot learning mainly focus on image classification while in this paper we focus on object detection. The initial explorations in few-shot object detection tend to simulate a classification scenario by using the positive proposals in images with respect to certain object class while discarding the negative proposals of that class. Negatives, especially hard negatives, however, are essential to the embedding space learning in few-shot object detection. In this paper, we restore the negative information in few-shot object detection by introducing a new negative- and positive-representative based metric learning framework and a new inference scheme with negative and positive representatives. We build our work on a recent few-shot pipeline RepMet with several new modules to encode negative information for both training and testing. Extensive experiments on ImageNet-LOC and PASCAL VOC show our method substantially improves the state-of-the-art few-shot object detection solutions. Our code is available at https://github.com/yang-yk/NP-RepMet.