Interactive Context-Aware Network for RGB-T Salient Object Detection

TL;DR

This paper introduces ICANet, a novel multi-modal network that effectively fuses RGB and thermal images for improved salient object detection, outperforming existing methods by leveraging cross-modal and cross-scale features.

Contribution

The paper proposes a new Interactive Context-Aware Network with modules for hybrid feature fusion, multi-scale attention, and a multi-supervised loss to enhance RGB-T salient object detection.

Findings

ICANet outperforms state-of-the-art RGB-T SOD methods.

The hybrid feature fusion effectively combines RGB and thermal features.

Multi-scale attention improves the accuracy of salient object localization.

Abstract

Salient object detection (SOD) focuses on distinguishing the most conspicuous objects in the scene. However, most related works are based on RGB images, which lose massive useful information. Accordingly, with the maturity of thermal technology, RGB-T (RGB-Thermal) multi-modality tasks attain more and more attention. Thermal infrared images carry important information which can be used to improve the accuracy of SOD prediction. To accomplish it, the methods to integrate multi-modal information and suppress noises are critical. In this paper, we propose a novel network called Interactive Context-Aware Network (ICANet). It contains three modules that can effectively perform the cross-modal and cross-scale fusions. We design a Hybrid Feature Fusion (HFF) module to integrate the features of two modalities, which utilizes two types of feature extraction. The Multi-Scale Attention Reinforcement (MSAR) and Upper Fusion (UF) blocks are responsible for the cross-scale fusion that converges different levels of features and generate the prediction maps. We also raise a novel Context-Aware Multi-Supervised Network (CAMSNet) to calculate the content loss between the prediction and the ground truth (GT). Experiments prove that our network performs favorably against the state-of-the-art RGB-T SOD methods.