Few-Shot Object Detection via Variational Feature Aggregation

TL;DR

This paper introduces a novel meta-learning framework with variational feature aggregation for few-shot object detection, improving class-agnostic representation and robustness to support example variance.

Contribution

It proposes Class-Agnostic Aggregation and Variational Feature Aggregation methods to enhance few-shot detection performance and robustness.

Findings

Outperforms baseline by up to 16% on PASCAL VOC and COCO

Achieves 4% average improvement over previous state-of-the-art

Effective in reducing confusion between base and novel classes

Abstract

As few-shot object detectors are often trained with abundant base samples and fine-tuned on few-shot novel examples,the learned models are usually biased to base classes and sensitive to the variance of novel examples. To address this issue, we propose a meta-learning framework with two novel feature aggregation schemes. More precisely, we first present a Class-Agnostic Aggregation (CAA) method, where the query and support features can be aggregated regardless of their categories. The interactions between different classes encourage class-agnostic representations and reduce confusion between base and novel classes. Based on the CAA, we then propose a Variational Feature Aggregation (VFA) method, which encodes support examples into class-level support features for robust feature aggregation. We use a variational autoencoder to estimate class distributions and sample variational features from distributions that are more robust to the variance of support examples. Besides, we decouple classification and regression tasks so that VFA is performed on the classification branch without affecting object localization. Extensive experiments on PASCAL VOC and COCO demonstrate that our method significantly outperforms a strong baseline (up to 16\%) and previous state-of-the-art methods (4\% in average). Code will be available at: \url{https://github.com/csuhan/VFA}