PS-TTL: Prototype-based Soft-labels and Test-Time Learning for Few-shot Object Detection

TL;DR

This paper introduces PS-TTL, a novel framework for few-shot object detection that combines prototype-based soft-labeling with test-time learning to improve detection performance with limited data.

Contribution

The paper proposes a new FSOD framework integrating a test-time learning module and a prototype-based soft-label strategy, enhancing detection accuracy with few samples.

Findings

Achieves state-of-the-art results on VOC and COCO benchmarks.

Effectively utilizes low-confidence pseudo-labels to improve detection.

Demonstrates significant performance gains over existing methods.

Abstract

In recent years, Few-Shot Object Detection (FSOD) has gained widespread attention and made significant progress due to its ability to build models with a good generalization power using extremely limited annotated data. The fine-tuning based paradigm is currently dominating this field, where detectors are initially pre-trained on base classes with sufficient samples and then fine-tuned on novel ones with few samples, but the scarcity of labeled samples of novel classes greatly interferes precisely fitting their data distribution, thus hampering the performance. To address this issue, we propose a new framework for FSOD, namely Prototype-based Soft-labels and Test-Time Learning (PS-TTL). Specifically, we design a Test-Time Learning (TTL) module that employs a mean-teacher network for self-training to discover novel instances from test data, allowing detectors to learn better representations and classifiers for novel classes. Furthermore, we notice that even though relatively low-confidence pseudo-labels exhibit classification confusion, they still tend to recall foreground. We thus develop a Prototype-based Soft-labels (PS) strategy through assessing similarities between low-confidence pseudo-labels and category prototypes as soft-labels to unleash their potential, which substantially mitigates the constraints posed by few-shot samples. Extensive experiments on both the VOC and COCO benchmarks show that PS-TTL achieves the state-of-the-art, highlighting its effectiveness. The code and model are available at https://github.com/gaoyingjay/PS-TTL.