Alignment-Free RGBT Salient Object Detection: Semantics-guided Asymmetric Correlation Network and A Unified Benchmark

TL;DR

This paper introduces SACNet, a novel alignment-free RGBT salient object detection method that effectively models cross-modal correlations in unaligned image pairs, supported by a new real-world dataset UVT2000.

Contribution

The paper proposes a semantics-guided asymmetric correlation network and a unified benchmark dataset for unaligned RGBT salient object detection, addressing a key challenge in the field.

Findings

SACNet outperforms existing methods on aligned and unaligned datasets.

The UVT2000 dataset enables research on real-world unaligned RGBT SOD.

The proposed modules effectively model cross-modal correlations without manual alignment.

Abstract

RGB and Thermal (RGBT) Salient Object Detection (SOD) aims to achieve high-quality saliency prediction by exploiting the complementary information of visible and thermal image pairs, which are initially captured in an unaligned manner. However, existing methods are tailored for manually aligned image pairs, which are labor-intensive, and directly applying these methods to original unaligned image pairs could significantly degrade their performance. In this paper, we make the first attempt to address RGBT SOD for initially captured RGB and thermal image pairs without manual alignment. Specifically, we propose a Semantics-guided Asymmetric Correlation Network (SACNet) that consists of two novel components: 1) an asymmetric correlation module utilizing semantics-guided attention to model cross-modal correlations specific to unaligned salient regions; 2) an associated feature sampling module to sample relevant thermal features according to the corresponding RGB features for multi-modal feature integration. In addition, we construct a unified benchmark dataset called UVT2000, containing 2000 RGB and thermal image pairs directly captured from various real-world scenes without any alignment, to facilitate research on alignment-free RGBT SOD. Extensive experiments on both aligned and unaligned datasets demonstrate the effectiveness and superior performance of our method. The dataset and code are available at https://github.com/Angknpng/SACNet.