Free Lunch for Co-Saliency Detection: Context Adjustment

TL;DR

This paper introduces a large, high-quality dataset called CAT for co-saliency detection, created using a novel context adjustment method and a cost-free data synthesis procedure, significantly improving model performance.

Contribution

The paper proposes a new dataset and a counterfactual training approach with context adjustment, addressing training-testing inconsistency in co-saliency detection.

Findings

CAT dataset improves state-of-the-art models by 5-25%

The GCP procedure enables effective data augmentation without additional labeling

Extensive experiments validate the dataset's benefits

Abstract

We unveil a long-standing problem in the prevailing co-saliency detection systems: there is indeed inconsistency between training and testing. Constructing a high-quality co-saliency detection dataset involves time-consuming and labor-intensive pixel-level labeling, which has forced most recent works to rely instead on semantic segmentation or saliency detection datasets for training. However, the lack of proper co-saliency and the absence of multiple foreground objects in these datasets can lead to spurious variations and inherent biases learned by models. To tackle this, we introduce the idea of counterfactual training through context adjustment and propose a "cost-free" group-cut-paste (GCP) procedure to leverage off-the-shelf images and synthesize new samples. Following GCP, we collect a novel dataset called Context Adjustment Training (CAT). CAT consists of 33,500 images, which is four times larger than the current co-saliency detection datasets. All samples are automatically annotated with high-quality mask annotations, object categories, and edge maps. Extensive experiments on recent benchmarks are conducted, show that CAT can improve various state-of-the-art models by a large margin (5% ~ 25%). We hope that the scale, diversity, and quality of our dataset can benefit researchers in this area and beyond. Our dataset will be publicly accessible through our project page.