Meta R-CNN : Towards General Solver for Instance-level Few-shot Learning

TL;DR

Meta R-CNN introduces a meta-learning approach over RoI features, enabling Faster/Mask R-CNN to effectively perform few-shot object detection and segmentation in complex scenes with multiple objects.

Contribution

It extends Faster/Mask R-CNN with a novel meta-learning over RoI features and a Predictor-head Remodeling Network for improved few-shot detection and segmentation.

Findings

Achieves state-of-the-art results in few-shot object detection.

Significantly improves few-shot object segmentation performance.

Demonstrates flexibility and generality across tasks.

Abstract

Resembling the rapid learning capability of human, few-shot learning empowers vision systems to understand new concepts by training with few samples. Leading approaches derived from meta-learning on images with a single visual object. Obfuscated by a complex background and multiple objects in one image, they are hard to promote the research of few-shot object detection/segmentation. In this work, we present a flexible and general methodology to achieve these tasks. Our work extends Faster /Mask R-CNN by proposing meta-learning over RoI (Region-of-Interest) features instead of a full image feature. This simple spirit disentangles multi-object information merged with the background, without bells and whistles, enabling Faster /Mask R-CNN turn into a meta-learner to achieve the tasks. Specifically, we introduce a Predictor-head Remodeling Network (PRN) that shares its main backbone with Faster /Mask R-CNN. PRN receives images containing few-shot objects with their bounding boxes or masks to infer their class attentive vectors. The vectors take channel-wise soft-attention on RoI features, remodeling those R-CNN predictor heads to detect or segment the objects that are consistent with the classes these vectors represent. In our experiments, Meta R-CNN yields the state of the art in few-shot object detection and improves few-shot object segmentation by Mask R-CNN.