SMILe: Leveraging Submodular Mutual Information For Robust Few-Shot Object Detection

TL;DR

SMILe introduces a submodular mutual information framework for few-shot object detection, enhancing class discrimination, reducing forgetting, and improving performance and convergence speed across benchmarks.

Contribution

The paper proposes a novel SMILe framework that uses combinatorial mutual information functions to improve feature clustering and class separation in FSOD, applicable across various architectures.

Findings

Achieves up to 5.7% mAP improvement on VOC and 5.4% on COCO in 10- and 30-shot settings.

Demonstrates better retention of base class performance.

Provides up to 2x faster convergence compared to existing methods.

Abstract

Confusion and forgetting of object classes have been challenges of prime interest in Few-Shot Object Detection (FSOD). To overcome these pitfalls in metric learning based FSOD techniques, we introduce a novel Submodular Mutual Information Learning (SMILe) framework which adopts combinatorial mutual information functions to enforce the creation of tighter and discriminative feature clusters in FSOD. Our proposed approach generalizes to several existing approaches in FSOD, agnostic of the backbone architecture demonstrating elevated performance gains. A paradigm shift from instance based objective functions to combinatorial objectives in SMILe naturally preserves the diversity within an object class resulting in reduced forgetting when subjected to few training examples. Furthermore, the application of mutual information between the already learnt (base) and newly added (novel) objects ensures sufficient separation between base and novel classes, minimizing the effect of class confusion. Experiments on popular FSOD benchmarks, PASCAL-VOC and MS-COCO show that our approach generalizes to State-of-the-Art (SoTA) approaches improving their novel class performance by up to 5.7% (3.3 mAP points) and 5.4% (2.6 mAP points) on the 10-shot setting of VOC (split 3) and 30-shot setting of COCO datasets respectively. Our experiments also demonstrate better retention of base class performance and up to 2x faster convergence over existing approaches agnostic of the underlying architecture.