Referring Camouflaged Object Detection

TL;DR

This paper introduces the new task of referring camouflaged object detection, proposing a large dataset and a dual-branch framework that effectively segments specified camouflaged objects based on referring images.

Contribution

The paper presents the first large-scale dataset R2C7K and a novel dual-branch R2CNet framework for referring camouflaged object detection, advancing the ability to identify specific camouflaged targets.

Findings

R2CNet outperforms existing COD methods in segmenting specified camouflaged objects.

The dataset R2C7K covers 64 object categories in real-world scenarios.

The proposed modules improve the accuracy of identifying target objects.

Abstract

We consider the problem of referring camouflaged object detection (Ref-COD), a new task that aims to segment specified camouflaged objects based on a small set of referring images with salient target objects. We first assemble a large-scale dataset, called R2C7K, which consists of 7K images covering 64 object categories in real-world scenarios. Then, we develop a simple but strong dual-branch framework, dubbed R2CNet, with a reference branch embedding the common representations of target objects from referring images and a segmentation branch identifying and segmenting camouflaged objects under the guidance of the common representations. In particular, we design a Referring Mask Generation module to generate pixel-level prior mask and a Referring Feature Enrichment module to enhance the capability of identifying specified camouflaged objects. Extensive experiments show the superiority of our Ref-COD methods over their COD counterparts in segmenting specified camouflaged objects and identifying the main body of target objects. Our code and dataset are publicly available at https://github.com/zhangxuying1004/RefCOD.