Improving Dense Contrastive Learning with Dense Negative Pairs

TL;DR

This paper introduces DenseCL++, an improved dense contrastive learning method that enhances spatial feature representations, leading to better performance in multi-label classification, detection, and segmentation tasks.

Contribution

The paper proposes DenseCL++, a novel training scheme and objective function for dense contrastive learning, with extensive ablations to understand key techniques and achieve state-of-the-art results.

Findings

3.5% and 4% mAP improvements on COCO multi-label classification

1.8% and 0.7% mIoU improvements on segmentation tasks

Effective techniques for forming dense negative pairs and auxiliary tasks

Abstract

Many contrastive representation learning methods learn a single global representation of an entire image. However, dense contrastive representation learning methods such as DenseCL (Wang et al., 2021) can learn better representations for tasks requiring stronger spatial localization of features, such as multi-label classification, detection, and segmentation. In this work, we study how to improve the quality of the representations learned by DenseCL by modifying the training scheme and objective function, and propose DenseCL++. We also conduct several ablation studies to better understand the effects of: (i) various techniques to form dense negative pairs among augmentations of different images, (ii) cross-view dense negative and positive pairs, and (iii) an auxiliary reconstruction task. Our results show 3.5% and 4% mAP improvement over SimCLR (Chen et al., 2020a) andDenseCL in COCO multi-label classification. In COCO and VOC segmentation tasks, we achieve 1.8% and 0.7% mIoU improvements over SimCLR, respectively.