Summarize and Search: Learning Consensus-aware Dynamic Convolution for Co-Saliency Detection

TL;DR

This paper introduces a novel consensus-aware dynamic convolution model for co-saliency detection, explicitly modeling the 'summarize and search' process to improve robustness and scalability, validated by experiments on benchmark datasets.

Contribution

The paper proposes a new dynamic convolution approach that explicitly models consensus summarization and object searching, enhancing co-saliency detection performance.

Findings

Outperforms existing methods on four benchmark datasets.

Effective consensus feature summarization using self-attention.

Improved robustness and scalability in co-saliency detection.

Abstract

Humans perform co-saliency detection by first summarizing the consensus knowledge in the whole group and then searching corresponding objects in each image. Previous methods usually lack robustness, scalability, or stability for the first process and simply fuse consensus features with image features for the second process. In this paper, we propose a novel consensus-aware dynamic convolution model to explicitly and effectively perform the "summarize and search" process. To summarize consensus image features, we first summarize robust features for every single image using an effective pooling method and then aggregate cross-image consensus cues via the self-attention mechanism. By doing this, our model meets the scalability and stability requirements. Next, we generate dynamic kernels from consensus features to encode the summarized consensus knowledge. Two kinds of kernels are generated in a supplementary way to summarize fine-grained image-specific consensus object cues and the coarse group-wise common knowledge, respectively. Then, we can effectively perform object searching by employing dynamic convolution at multiple scales. Besides, a novel and effective data synthesis method is also proposed to train our network. Experimental results on four benchmark datasets verify the effectiveness of our proposed method. Our code and saliency maps are available at \url{https://github.com/nnizhang/CADC}.