Long-Tailed Object Detection Pre-training: Dynamic Rebalancing Contrastive Learning with Dual Reconstruction

TL;DR

This paper proposes a novel pre-training framework for object detection that uses dynamic rebalancing and dual reconstruction to address data imbalance and simplicity bias, significantly improving tail class detection performance.

Contribution

The paper introduces 2DRCL, a pre-training method combining holistic-local contrastive learning with dynamic rebalancing and dual reconstruction to enhance long-tailed object detection.

Findings

Improves mAP for tail classes on COCO and LVIS datasets.

Effectively balances representation of underrepresented classes during pre-training.

Demonstrates superior performance over existing pre-training methods in long-tailed detection tasks.

Abstract

Pre-training plays a vital role in various vision tasks, such as object recognition and detection. Commonly used pre-training methods, which typically rely on randomized approaches like uniform or Gaussian distributions to initialize model parameters, often fall short when confronted with long-tailed distributions, especially in detection tasks. This is largely due to extreme data imbalance and the issue of simplicity bias. In this paper, we introduce a novel pre-training framework for object detection, called Dynamic Rebalancing Contrastive Learning with Dual Reconstruction (2DRCL). Our method builds on a Holistic-Local Contrastive Learning mechanism, which aligns pre-training with object detection by capturing both global contextual semantics and detailed local patterns. To tackle the imbalance inherent in long-tailed data, we design a dynamic rebalancing strategy that adjusts the sampling of underrepresented instances throughout the pre-training process, ensuring better representation of tail classes. Moreover, Dual Reconstruction addresses simplicity bias by enforcing a reconstruction task aligned with the self-consistency principle, specifically benefiting underrepresented tail classes. Experiments on COCO and LVIS v1.0 datasets demonstrate the effectiveness of our method, particularly in improving the mAP/AP scores for tail classes.