Modality-Agnostic Prompt Learning for Multi-Modal Camouflaged Object Detection

TL;DR

This paper introduces a modality-agnostic prompt learning framework for camouflaged object detection that enhances cross-modal generalization and accuracy by leveraging a unified prompt approach with the Segment Anything Model.

Contribution

It proposes a novel, scalable prompt-based method that adapts to arbitrary auxiliary modalities for improved camouflaged object detection performance.

Findings

Significant performance gains on RGB-Depth, RGB-Thermal, and RGB-Polarization datasets.

Effective generalization across multiple modalities with a unified prompt framework.

Improved boundary accuracy with a lightweight Mask Refine Module.

Abstract

Camouflaged Object Detection (COD) aims to segment objects that blend seamlessly into complex backgrounds, with growing interest in exploiting additional visual modalities to enhance robustness through complementary information. However, most existing approaches generally rely on modality-specific architectures or customized fusion strategies, which limit scalability and cross-modal generalization. To address this, we propose a novel framework that generates modality-agnostic multi-modal prompts for the Segment Anything Model (SAM), enabling parameter-efficient adaptation to arbitrary auxiliary modalities and significantly improving overall performance on COD tasks. Specifically, we model multi-modal learning through interactions between a data-driven content domain and a knowledge-driven prompt domain, distilling task-relevant cues into unified prompts for SAM decoding. We further introduce a lightweight Mask Refine Module to calibrate coarse predictions by incorporating fine-grained prompt cues, leading to more accurate camouflaged object boundaries. Extensive experiments on RGB-Depth, RGB-Thermal, and RGB-Polarization benchmarks validate the effectiveness and generalization of our modality-agnostic framework.