UIFormer: A Unified Transformer-based Framework for Incremental Few-Shot Object Detection and Instance Segmentation

TL;DR

UIFormer is a Transformer-based framework that effectively handles incremental few-shot object detection and segmentation, maintaining high performance on new and old classes without access to previous training data.

Contribution

It introduces a unified two-stage incremental learning framework with classifier selection and knowledge distillation, advancing the state-of-the-art in incremental few-shot detection and segmentation.

Findings

Outperforms existing methods on COCO and LVIS datasets

Effectively mitigates overfitting on novel classes

Prevents catastrophic forgetting of base classes

Abstract

This paper introduces a novel framework for unified incremental few-shot object detection (iFSOD) and instance segmentation (iFSIS) using the Transformer architecture. Our goal is to create an optimal solution for situations where only a few examples of novel object classes are available, with no access to training data for base or old classes, while maintaining high performance across both base and novel classes. To achieve this, We extend Mask-DINO into a two-stage incremental learning framework. Stage 1 focuses on optimizing the model using the base dataset, while Stage 2 involves fine-tuning the model on novel classes. Besides, we incorporate a classifier selection strategy that assigns appropriate classifiers to the encoder and decoder according to their distinct functions. Empirical evidence indicates that this approach effectively mitigates the over-fitting on novel classes learning. Furthermore, we implement knowledge distillation to prevent catastrophic forgetting of base classes. Comprehensive evaluations on the COCO and LVIS datasets for both iFSIS and iFSOD tasks demonstrate that our method significantly outperforms state-of-the-art approaches.