Multi-Perspective Data Augmentation for Few-shot Object Detection

TL;DR

This paper introduces a Multi-Perspective Data Augmentation framework for few-shot object detection, enhancing synthetic sample diversity by considering foreground and background relationships, leading to significant performance improvements.

Contribution

The novel MPAD framework combines in-context learning, prompt aggregation, and background sampling to improve synthetic data quality for FSOD.

Findings

Achieves 17.5% nAP50 improvement on PASCAL VOC

Effectively enhances synthetic sample diversity

Outperforms traditional augmentation methods

Abstract

Recent few-shot object detection (FSOD) methods have focused on augmenting synthetic samples for novel classes, show promising results to the rise of diffusion models. However, the diversity of such datasets is often limited in representativeness because they lack awareness of typical and hard samples, especially in the context of foreground and background relationships. To tackle this issue, we propose a Multi-Perspective Data Augmentation (MPAD) framework. In terms of foreground-foreground relationships, we propose in-context learning for object synthesis (ICOS) with bounding box adjustments to enhance the detail and spatial information of synthetic samples. Inspired by the large margin principle, support samples play a vital role in defining class boundaries. Therefore, we design a Harmonic Prompt Aggregation Scheduler (HPAS) to mix prompt embeddings at each time step of the generation process in diffusion models, producing hard novel samples. For foreground-background relationships, we introduce a Background Proposal method (BAP) to sample typical and hard backgrounds. Extensive experiments on multiple FSOD benchmarks demonstrate the effectiveness of our approach. Our framework significantly outperforms traditional methods, achieving an average increase of $17.5\%$ in nAP50 over the baseline on PASCAL VOC. Code is available at https://github.com/nvakhoa/MPAD.