iFS-RCNN: An Incremental Few-shot Instance Segmenter

TL;DR

This paper introduces iFS-RCNN, an incremental few-shot instance segmentation method that extends Mask-RCNN with Bayesian class classification and uncertainty-guided bounding box prediction, achieving significant performance improvements.

Contribution

It proposes novel Bayesian classification and uncertainty-guided bounding box refinement in Mask-RCNN for incremental few-shot segmentation, with new loss functions and evaluation on COCO and LVIS datasets.

Findings

+6 AP on new classes in COCO

+16 AP on old classes in COCO

First evaluation on LVIS for this setting

Abstract

This paper addresses incremental few-shot instance segmentation, where a few examples of new object classes arrive when access to training examples of old classes is not available anymore, and the goal is to perform well on both old and new classes. We make two contributions by extending the common Mask-RCNN framework in its second stage -- namely, we specify a new object class classifier based on the probit function and a new uncertainty-guided bounding-box predictor. The former leverages Bayesian learning to address a paucity of training examples of new classes. The latter learns not only to predict object bounding boxes but also to estimate the uncertainty of the prediction as guidance for bounding box refinement. We also specify two new loss functions in terms of the estimated object-class distribution and bounding-box uncertainty. Our contributions produce significant performance gains on the COCO dataset over the state of the art -- specifically, the gain of +6 on the new classes and +16 on the old classes in the AP instance segmentation metric. Furthermore, we are the first to evaluate the incremental few-shot setting on the more challenging LVIS dataset.