MCFNet: Multi-Scale Contextual Fusion Network for Salient Object Detection in Optical Remote Sensing Images

TL;DR

This paper introduces MCFNet, a new deep learning network for detecting salient objects in optical remote sensing images, improving accuracy and boundary delineation.

Contribution

The novel MCFNet combines a Semantic-Aware Attention Module and Contextual Interconnection Module for enhanced salient object detection in remote sensing.

Findings

MCFNet outperforms existing methods on standard ORSI-SOD benchmark datasets.

The network achieves robust and efficient performance in challenging remote sensing scenarios.

MCFNet improves localization accuracy and boundary delineation for complex and varying-scale objects.

Abstract

The rapid advancement of deep learning has catalyzed progress in salient object detection (SOD), extending its impact to the domain of optical remote sensing images (ORSIs). Despite increasing attention, salient object detection for optical remote sensing images (ORSI-SOD) remains highly challenging due to the intrinsic complexities of remote sensing scenes. In particular, severe variations in object scale and quantity, cluttered backgrounds, and irregular object morphologies significantly hinder accurate target localization and boundary delineation. In response to these challenges, we introduce the Multi-scale Contextual Fusion Network (MCFNet) for ORSI-SOD. MCFNet incorporates a Semantic-Aware Attention Module (SAM), which provides explicit semantic guidance during feature extraction. By producing preliminary semantic masks, SAM enables the network to capture long-range contextual dependencies, thereby enhancing localization accuracy for salient objects exhibiting substantial scale variation and structural complexity. In addition, MCFNet integrates a Contextual Interconnection Module (CIM), which promotes effective fusion of local and global contextual features. By facilitating cross-layer interactions and adopting a multiscale refinement strategy, CIM enriches texture representations while suppressing background interference, leading to smoother object boundaries and more precise delineation of salient regions. Extensive evaluations conducted on three standard ORSI-SOD benchmark datasets demonstrate the superior performance of MCFNet compared to existing methods, highlighting its robustness and efficiency in handling challenging remote sensing scenarios.

Figures

Full text

1. Introduction

Salient Object Detection (SOD) [1] is a fundamental task in computer vision, aimed at emulating the human visual system’s ability to identify and localize the most visually prominent regions within a scene. In recent years, SOD has found widespread applications in various domains, including image segmentation [2,3], image quality assessment [4], visual target tracking [5,6], and content-aware image editing [7]. Optical Remote Sensing Images (ORSIs) are captured by satellites and onboard antenna sensors. Salient object detection in remote sensing images (ORSI-SOD) [8] can effectively identify the most salient target areas in ORSIs with the help of in-depth learning and detailed analysis of image features in both spatial and spectral dimensions. However, the ORSI-SOD task faces several challenges due to the high-altitude and long-distance nature of image capture. In particular, the variability in target numbers and scales is a significant issue, as a single image may contain numerous targets of varying sizes. Furthermore, the complexity of the background environment poses further challenges, as it can obscure the detection of salient targets. Irregular topological features within ORSIs also contribute to the difficulty of accurately identifying prominent targets.

Recent years have witnessed notable advances in salient object detection for natural scene images (NSI-SOD), leading to the development of numerous effective methods. These methods can be broadly divided into main types: traditional methods and deep learning-based methods. Traditional methods [9,10] rely primarily on model-driven feature extraction [11] and classical machine learning algorithms [12]. Although grounded in solid theoretical foundations, these approaches frequently encounter difficulties in delivering precise detection results when faced with intricate or cluttered scenes because of limited generalization capabilities and poor adaptability to diverse background conditions. In contrast, the advent of convolutional neural networks (CNNs) has led to the development of data-driven deep learning models [13,14] that have demonstrated remarkable performance in NSI-SOD tasks. These approaches leverage large-scale annotated datasets to learn robust and discriminative feature representations. For instance, the PA-KRN [15] adopts a hierarchical architecture to enhance salient object detection in natural images, while the SUCA [16] further improves multiscale target perception through stacked interaction networks. Such deep learning methods have consistently outperformed traditional methods in accuracy and robustness. However, directly applying NSI-SOD models to ORSI-SOD tasks is not trivial [17]. ORSIs differ markedly from NSIs in terms of spatial distribution, viewpoint, resolution, and overall scene composition. In addition, they exhibit greater variability in object scale, shape, and appearance. These domain-specific challenges limit the effectiveness of NSI-SOD models when transferred to remote sensing scenarios. As a result, there is a critical need for the development of specialized and adaptive methods tailored to the unique demands of ORSI-SOD.

In response to the limitations of transferring NSI-SOD methods to remote sensing scenarios, several approaches [18,19] specifically designed for ORSI-SOD have been proposed. For example, nested networks [20] generate high-level attention maps by integrating low-level cues with deeper features through a dense attention mechanism. However, these methods often overlook the impact of high-level semantic information on low-level feature representation during attention transfer, which compromises the refinement of object boundaries. ACCoNet [21] improves saliency detection by enabling cross-layer feature interaction and effectively extracting contextual semantic information. Despite its effectiveness in contextual modeling, it incurs substantial computational overhead. AESINet [22] employs separate modules to process high- and low-level features extracted from multiple CNN layers and captures adaptive edge information through perceptual mapping of self-pooled differences. Although these approaches are specifically tailored for ORSI-SOD, they remain limited when confronted with the compounded challenges of varying object quantities and scales, intricate background environments, and irregular object topologies.

Motivated by the limitations of existing approaches, we aim to tackle the persistent challenges in ORSI-SOD, such as background clutter, variable object scales and densities, and inherently low contrast between targets and their surroundings. In this study, we introduce a novel Multi-scale Contextual Fusion Network (MCFNet), which is specifically engineered to achieve robust and accurate saliency detection under complex and diverse conditions. MCFNet is constructed based on two complementary modules: the Semantic Attention Module (SAM) and the Contextual Interconnection Module (CIM). SAM plays a pivotal role in precise target localization. It integrates coarse semantic masks as initial priors, providing robust and explicit semantic guidance to subsequent layers. In contrast to conventional methods, which often falter in capturing long-range dependencies, SAM introduces a global semantic perception mechanism. This mechanism effectively overcomes the constraints imposed by local receptive fields, allowing the network to accurately localize salient regions, even when dealing with objects of varying scales and spatially dispersed distributions. Working in tandem with SAM, CIM is crucial for enhancing object saliency and suppressing background noise. Given that RSIs frequently contain background elements that resemble foreground targets in texture or color, the risk of false positives remains high. CIM addresses this issue by employing a multiscale refinement strategy, enabling deep, cross-layer feature interactions. Through the mining of inter-scale contextual dependencies, CIM effectively integrates high-level semantic cues with low-level texture details. To comprehensively evaluate the effectiveness of MCFNet, we perform extensive experiments on three widely used public ORSI-SOD benchmarks. Across all datasets and most evaluation metrics, MCFNet consistently outperforms 20 state-of-the-art methods, demonstrating clear superiority in both detection accuracy and robustness.

The main contributions of this work can be outlined as follows.

1.We introduce a Multi-scale Contextual Fusion Network (MCFNet) built upon an encoder–decoder framework, designed to effectively extract high-level semantic cues while promoting the integration of low-level detailed features with high-level contextual information via a cross-level interconnection strategy.2.We introduce a Semantic-Aware Attention Module (SAM), which leverages preliminary semantic masks to guide the extraction of deep semantic information in ORSIs, thereby enhancing the ability to localize salient targets under complex conditions.3.We design a Contextual Interconnection Module (CIM) that enriches texture representations across multiple scales by refining local features and adaptively fusing contextual information from adjacent layers.4.Comprehensive evaluations conducted on three datasets, ORSSD, EORSSD, and ORSI4199, show that our approach achieves superior performance compared to existing methods. Furthermore, ablation studies confirm the contribution and effectiveness of each proposed module.

2. Related Work

2.1. Salient Object Detection Methods in NSIs

The SOD task was originally developed for NSIs analysis. Over the past few years, NSI-SOD has witnessed substantial progress, with a wide range of methods proposed, spanning both traditional and deep learning approaches. Traditional NSI-SOD approaches are generally grouped into three categories: unsupervised methods [11,23,24,25], semi-supervised methods [26,27], and supervised methods [28]. Among these, unsupervised methods have historically dominated, typically relying on interpretable handcrafted features extracted through image preprocessing techniques, such as the sparse graph [11], the saliency tree [23], the maximal entropy random walk [24], and the regularized random walks ranking [25]. In contrast, relatively few traditional methods adopt semi-supervised or supervised strategies. For instance, Zhou et al. [26] developed a boundary-guided approach for pseudo-label generation and designed a semi-supervised learning framework based on a linear feedback control mechanism for iterative optimization. Liang et al. [28] applied supervised learning via support vector machines to select salient features and eliminate redundancies, thereby accelerating model training. Despite their contributions, traditional NSI-SOD methods struggle to generalize across the diverse and complex scenarios encountered in ORSIs.

CNN-based methods for NSI-SOD have overcome many of the limitations inherent in traditional approaches. Unlike traditional techniques, most CNN-based methods adopt a supervised learning paradigm, relying on large-scale annotated datasets to significantly enhance detection accuracy. Over the past years, several influential feature processing strategies have been introduced, including deep supervision [29], multi-scale interaction [14,30], and edge-aware feature enhancement [13]. Huang et al. [29] were among the first to incorporate deep supervision into NSI-SOD frameworks, inspiring a wave of subsequent research exploring its potential. PoolNet [30] utilizes a combination of top-down and bottom-up pathways to extract and integrate multi-scale contextual information. GateNet [14] introduces collapsed spatial pyramid pooling to aggregate spatial feature maps at varying receptive fields, thereby enhancing multi-scale representation. Meanwhile, EGNet [13] focuses on capturing edge features, improving the precision of object localization by highlighting object boundaries more effectively. Although deep learning-based NSI-SOD methods have achieved remarkable success, applying them directly to ORSI-SOD tasks poses considerable challenges. The fundamental differences between NSIs and ORSIs pose substantial challenges, limiting the generalizability and effectiveness of NSI-SOD models in remote sensing contexts.

2.2. Salient Object Detection Methods in ORSIs

Compared with the NSI-SOD task, ORSI-SOD faces more complex and unique challenges. To tackle these challenges, CNN-based models for ORSI-SOD offer promising solutions owing to their strong abilities in feature representation and learning. Specifically, Ma et al. explored efficient segmentation solutions for SAR images from different technical perspectives, proposing a fast segmentation method based on deep task-specific superpixel sampling and soft graph convolution [31] as well as a task-specific region merging strategy for rapid segmentation [32]. Li et al. [20] designed an LV-shaped network architecture, where an L-shaped pyramid module processes inputs at different resolutions, and a V-shaped nested connections structure precisely identifies the position and boundaries of salient objects. Luo et al. [33] further enhanced feature learning by adopting convolution kernels of diverse shapes, which enables it to capture a wide range of salient target features, including topology and orientation, thereby improving the model’s adaptability to complex target morphologies. Tu et al. [34] combined local features extracted via pyramid pooling with global context to construct a strong multi-scale regional representation. Gu et al. [35] introduced an edge-guided branching structure to generate rich boundary information, leading to accurate localization and highlighting of salient regions under boundary label supervision. Despite the remarkable progress made using deep neural networks in ORSI-SOD, CNN-based models face inherent limitations in global modeling capabilities, which often results in difficulty detecting objects across varying scales simultaneously. This problem is especially prominent in RSIs, which usually contain richer and more diverse scales and target morphologies, putting forward higher requirements on the global perception ability of the model.

Transformers [36] excel at capturing long-range interactions, enabling effective extraction of global context from visual data. Consequently, recent studies have increasingly incorporated Transformers into the ORSI-SOD task to enhance global modeling capabilities. Specifically, Zhang et al. [37] designed a dense attention flow structure, which facilitates the efficient connection and fusion of features at multiple levels through an attention mechanism. Zhou et al. [38] introduced a U-shaped architecture, incorporating edge-aware positional attention units to strengthen edge cue extraction, thereby improving the detection accuracy of salient targets. Inspired by this, we propose an efficient global–local perception network based on contextual interactions. Within this network, global information, which is extracted through semantic-aware attention mechanisms, is progressively integrated with local texture details via contextual interactions at every layer. This novel architecture enhances the model’s ability to model complex global dependencies while enabling accurate localization and refinement of fine-grained local features.

3. Method

In this study, we present a comprehensive analysis of the proposed Multi-scale Contextual Fusion Network (MCFNet), as shown in Figure 1. MCFNet adopts an encoder–decoder framework and integrates two key components: the Semantic-aware Attention Module (SAM) and the Contextual Interconnection Module (CIM). In Section 3.1, we provide a brief overview of the MCFNet framework. Subsequently, in Section 3.2 and Section 3.3, we offer an in-depth discussion of these two core components. Finally, in Section 3.4, we elaborate on the design of the decoder and the hybrid loss function.

3.1. Network Overview

Figure 1 presents the architecture of MCFNet, which adopts a generalized encoder–decoder framework comprising an encoder, a semantic-aware attention module (SAM), a contextual interconnection module (CIM), and a decoder. For feature extraction, we employ Res2Net-50 [36] as the backbone. The encoder is structured into five blocks, which capture multi-scale features encompassing both low-level and high-level information. The output of each block can be represented as $[eqn]$ , $[eqn]$ , where $[eqn]$ and $[eqn]$ are $[eqn]$ and $[eqn]$ . Notably, the high-level feature encapsulates rich semantic information crucial for ORSI-SOD. For high-level feature $[eqn]$ , we employ SAM, which enhances semantic detail and suppresses background noise, thereby improving the localization of salient regions in complex scenes. In contrast, the low-level features $[eqn]$ , $[eqn]$ , $[eqn]$ , and $[eqn]$ possess abundant texture information but lack global context. To address this, the CIM aggregates cross-scale features from adjacent blocks, facilitating enhanced interaction between low-level and high-level representations. Finally, the decoder consists of five blocks that progressively reconstruct the resolution. The hybrid loss function is employed to enable deep supervision and produce the final output prediction maps.

3.2. Semantic-Aware Attention Module

When processing remote sensing images, salient objects exhibit substantial variability in number and scale and are often obscured by complex backgrounds, presenting significant challenges for accurate target identification and localization. To address these issues, most CNN-based methods typically employ deep convolutional stacks to extract features from increasingly large receptive fields. However, these methods tend to lose critical target information during progressive downsampling, largely due to the inherent limitations of CNNs in global context modeling. Although CNNs effectively capture local image content, their extracted features are insufficient to suppress background noise and achieve precise target localization. In contrast, the self-attention mechanism has gained recognition for its superior ability to model global dependencies [39]. Motivated by this insight, we propose the Semantic-aware Attention Module (SAM), which leverages a preliminary semantic mask to guide feature extraction. SAM enhances critical global information, thereby facilitating robust and accurate target localization by efficiently capturing long-range dependencies and overcoming the limitations of local feature extraction.

The high-level feature $[eqn]$ extracted from the encoder’s fifth layer contains essential semantic information crucial for object detection. To enhance the localization of objects, we first process $[eqn]$ through a series of operations—including a $[eqn]$ convolution, batch normalization, ReLU activation, and sigmoid function—to generate a preliminary semantic mask $[eqn]$ . In this process, we preserve the preliminary semantic mask as continuous values within the $[eqn]$ interval rather than binarizing them. Subsequently, we input the preliminary mask $[eqn]$ along with the original $[eqn]$ into our proposed SAM. As depicted in Figure 2, within the SAM, a bilinear fusion scheme is first employed to compute the attention matrix. Specifically, the input feature $[eqn]$ is efficiently decomposed into two intermediate features $[eqn]$ and $[eqn]$ via pointwise convolution. Then, the depthwise separable convolution layer is used to generate a query vector $[eqn]$ and a key vector $[eqn]$ . This effective combination of pointwise and depthwise separable convolutions preserves powerful feature extraction while significantly reducing computational complexity. To emphasize salient targets in the remote sensing image, we apply the preliminary semantic mask $[eqn]$ to both Q and K via element-wise multiplication. This operation accentuates the target region while suppressing extraneous information, yielding the semantic-aware query $[eqn]$ and key $[eqn]$ , which can be formulated as:

$$[eqn]$$ $$[eqn]$$

where $[eqn]$ is the depthwise separable convolution, $[eqn]$ is the pointwise convolution, and ⊙ represents the element-wise multiplication. Subsequently, $[eqn]$ and the transposed $[eqn]$ undergo matrix multiplication to produce an attention map $[eqn]$ , which quantifies the association strength between different spatial locations. The computational results are presented below.

$$[eqn]$$

where ⊗ is the matrix multiplication, and $[eqn]$ denotes the softmax layer. Based on these operations, the resulting attention matrix identifies effectively encoded spatial information from a global perspective, thereby capturing and leveraging semantic cues.

To construct the value matrix V in the self-attention mechanism, we apply a depthwise separable convolution layer to the high-level feature $[eqn]$ to efficiently refine the target feature representation. Owing to its inherently small receptive field, depthwise separable convolution effectively emphasizes local feature details. In parallel, feature $[eqn]$ is processed through three sequential dilated convolution layers with expansion rates of 2, 4, and 2, respectively, to broaden the receptive field and capture long-range contextual information. By integrating these two strategies, the value matrix V simultaneously captures local details and global context, which is computed as:

$$[eqn]$$

where ⊕ represents the element-wise summation and $[eqn]$ and $[eqn]$ denote dilated convolutions with dilation rates of 2 and 4. In the final stage, we multiply the attention matrix A with the value matrix V and obtain the semantic-aware feature vector $[eqn]$ . Then, the semantic-aware feature $[eqn]$ is fused with the original feature vector $[eqn]$ to yield the final output $[eqn]$ of SAM, which can be formulated as follows:

$$[eqn]$$

where $[eqn]$ is a learnable scaling parameter. Within the SAM, we employ a semantic-aware attention mechanism designed to amplify the salient object’s semantic representation while attenuating noise interference, thereby enabling high-precision target localization.

3.3. Contextual Interconnection Module

By employing SAM, we improve salient object localization by emphasizing global semantic cues. However, it may overlook fine-grained details essential for precise segmentation. In contrast, low-level features $[eqn]$ and $[eqn]$ encode high-resolution textures and subtle spatial details crucial for delineating accurate object boundaries. Recognizing that effective segmentation requires a balanced interplay between semantic-rich high-level context and detailed low-level features, we propose the Contextual Interaction Module (CIM). CIM integrates adjacent-level features, facilitating multi-scale information interaction via a dedicated Hierarchical Refinement Unit, which restores detailed feature representations. Additionally, an adaptive weighted unit is introduced to further enhance local feature representations, contributing to improved segmentation precision and stability.

In Figure 3, CIM comprises three primary branches, CIM-2, CIM-3, and CIM-4. Each of these branches integrates contextual information derived from its corresponding local and adjacent feature scales. Notably, CIM-1 incorporates only two branches. Accordingly, the operation of CIM can be abstracted as a generalized function $[eqn]$ , which can be formulated as follows:

$$[eqn]$$

where $[eqn]$ denotes the output features from CIM and represents the fusion of cross-level features from the previous, current, and subsequent feature blocks.

The localization branch incorporates two core components: the Hierarchical Refinement Unit (HRU) and the Adaptive Weighted Adjustment Unit (WAU). Initially, the input feature $[eqn]$ is processed through the HRU, which utilizes asynchronous convolution techniques to extract features and optimize multi-scale details. To comprehensively capture local features at multiple scales, we construct four parallel branches, each tailored to a specific receptive scale. The first branch employs a single $[eqn]$ convolutional layer with residual connection to preserve critical input information. The subsequent three branches expand the receptive fields to capture broader spatial contexts by employing asymmetric convolutions with kernel sizes of $[eqn]$ and $[eqn]$ , where t is set to 3, 5, and 7. Specifically, the $[eqn]$ convolution enhances sensitivity to vertically oriented structures by extending the kernel height, while the $[eqn]$ convolution captures horizontally oriented details by increasing kernel width. This dual-directional receptive field expansion enables each branch to comprehensively encode contextual information at its respective scale. Following these processing steps, the four branches produce output feature maps $[eqn]$ , $[eqn]$ , $[eqn]$ , and $[eqn]$ . The specific computational procedure is shown below:

$$[eqn]$$

where $[eqn]$ denotes the convolution operation using a $[eqn]$ kernel, while $[eqn]$ and $[eqn]$ indicate convolutions performed separately with kernels of dimensions $[eqn]$ and $[eqn]$ , respectively. Subsequently, the outputs $[eqn]$ , $[eqn]$ , $[eqn]$ , and $[eqn]$ generated by the HRU are fed into WAU. For each feature $[eqn]$ , a channel attention mechanism independently calculates the channel attention $[eqn]$ , enhancing the expressiveness along the channel dimension. The attention maps $[eqn]$ are then used to derive corresponding weights $[eqn]$ , which are subsequently applied through element-wise multiplication to their respective feature maps, enabling adaptive weighting and refinement. All weighted feature maps are aggregated through summation to produce the integrated output feature $[eqn]$ . The computational procedure is summarized as follows:

$$[eqn]$$

where $[eqn]$ is a linear layer and $[eqn]$ denotes channel attention operation. Thereafter, spatial information from $[eqn]$ is extracted via the spatial attention mechanism, yielding the spatially enhanced feature $[eqn]$ .

The two adjacent branches effectively capture features at different levels. Specifically, the lower-level feature $[eqn]$ is first rescaled to match the spatial dimensions of the current feature $[eqn]$ . In order to enrich the local representations, we obtain the refined feature through the depthwise separable convolution and spatial attention mechanism to realize the accurate segmentation. The process can be formulated as:

$$[eqn]$$

where $[eqn]$ is the output of the adjacent lower-level branch and $[eqn]$ represents the max-pooling layer with a kernel size of $[eqn]$ and stride of 2. Similarly, the higher-level feature $[eqn]$ employs the same operation to refine the global information, which is formulated as follows:

$$[eqn]$$

where $[eqn]$ is the output of the adjacent higher-level branch and $[eqn]$ is the upsampling operation. Following these operations, we merge the output features from each branch with the current feature. The adjacent lower-level branch provides detailed texture information, whereas higher-level branches introduce global semantic context. The fusion process is described as follows:

$$[eqn]$$

In summary, CIM effectively connects and integrates contextual semantic information to facilitate accurate salient object segmentation.

3.4. Decoder and Loss Function

To obtain accurate salient objects, we employ the widely used deep supervision strategy during the decoding phase, enabling effective supervision [29] of intermediate salient maps. As shown in Figure 1, the decoder comprises five modules, each consisting of convolution layers followed by a deconvolution layer. Through this design, spatial resolution is progressively recovered, and feature representations are iteratively refined. Ultimately, this decoding process generates a salient projection map $[eqn]$ with dimensions $[eqn]$ , where pixel values range between 0 and 1.

During training, pixel-level supervision is applied to each decoder stage. We employed a hybrid loss ( $[eqn]$ ) function combining pixel-level weighted binary cross-entropy (BCE) loss with map-level weighted intersection-over-union (IoU) loss. Specifically, the pixel-level weighted BCE loss captures the discrepancy between the predicted and actual masks at each pixel, while the map-level weighted IoU loss evaluates the spatial agreement by calculating the intersection-over-union between the predicted and actual masks. The combined hybrid loss can be formally expressed as:

$$[eqn]$$

where G denotes the ground truth, and $[eqn]$ and $[eqn]$ represent the BCE loss and IoU loss, respectively.

4. Experiments and Results

4.1. Experiment Protocol

(1) Datasets. To assess the effectiveness of our MCFNet, extensive experiments were carried out on three benchmark ORSI-SOD datasets, i.e., ORSSD [20], EORSSD [37], and ORSI4199 [34]. The ORSSD dataset includes 800 images with pixel-level annotations, split into 600 for training and 200 for testing. The EORSSD dataset, an extension of ORSSD, contains 2000 high-resolution images with corresponding ground truths, typically divided into 1400 training and 600 testing samples. The ORSI4199 dataset is the latest and most challenging dataset in the ORSI-SOD field, comprising 2000 training and 2199 testing images. Its rich diversity and complex scenes pose significant challenges for detection models. For each dataset, training and evaluation are conducted using their designated splits.

(2) Implementation Details. Our MCFNet was trained and evaluated on a workstation equipped with an NVIDIA RTX 3090 GPU with 24 GB of memory. All input ORSIs were uniformly resized to $[eqn]$ for both the training and testing phases. To improve model robustness and dataset variability, standard augmentation techniques such as horizontal flipping and rotation were employed. The network was optimized using the Adam optimizer [40], with an initial learning rate of $[eqn]$ , which was decayed by a factor of 0.1 after 50 epochs. Training was conducted for 100 epochs with a batch size of 16.

(3) Evaluation Metrics. We thoroughly evaluated the proposed MCFNet against leading state-of-the-art approaches on three benchmark ORSI-SOD datasets, utilizing six standard evaluation metrics: S-measure ( $[eqn]$ ) [41], Mean Absolute Error (MAE, $[eqn]$ ), mean E-measures ( $[eqn]$ ) [42], maximum E-measure ( $[eqn]$ ), mean F-measure ( $[eqn]$ ) [43], and maximum F-measure ( $[eqn]$ ). Specifically, the S-measure captures the structural similarity between region-aware and object-aware predictions, which can be expressed as:

$$[eqn]$$

where S denotes the predicted salient map, G denotes the ground truth, and H and W denote the height and width, respectively. The MAE quantifies the average pixel-wise discrepancy between predicted saliency maps and ground truth. The mathematical expression is formulated as follows:

$$[eqn]$$

where $[eqn]$ and $[eqn]$ denote the object-aware and region-aware structural similarity measures, respectively. Both E-measure and F-measure are evaluated in terms of their mean and maximum values, offering a more complete view of performance. The E-measure combines pixel-level precision and object-level consistency to assess structural coherence and is defined as follows:

$$[eqn]$$

where $[eqn]$ represents the enhanced-alignment matrix. The F-measure offers a harmonic trade-off between precision and recall to reflect the overall segmentation accuracy and is defined as follows:

$$[eqn]$$

Additionally, we analyzed model performance using precision–recall (PR) curves, where curves closer to the top-right corner (1,1) indicate better performance in terms of both precision and recall.

4.2. Comparison with State-of-the-Art Methods

(1) Competing Methods. In this study, we present MCFNet and evaluate its effectiveness by benchmarking it against 20 leading methods on three widely used datasets, i.e., ORSSD, EORSSD, and ORSI4199, to comprehensively assess both accuracy and generalization capability. Our evaluation incorporates both qualitative and quantitative analyses. For comparative purposes, we selected eight prominent methods from the natural image salient object detection, including R3Net [44], PoolNet [30], EGNet [13], SUCA [16], U2Net [45], GateNet [14], MINet [46], and PA-KRN [15]. Meanwhile, we compared our approach with twelve specialized ORSI-SOD methods, including DAFNet [37], EMFINet [47], AGNet [48], MJRBM [34], ACCoNet [21], CorrNet [49], BAFS-Net [35], ERPNet [38], AESINet [22], SFANet [50], ADSTNet [51], and LSHNet [17]. To ensure fairness and consistency, all models were trained and tested under identical experimental conditions and parameter configurations. Furthermore, predictions from competing models were generated using publicly available implementations provided by their respective authors, ensuring the reliability and reproducibility of our results.

(2) Quantitative Comparison. Table 1 presents a comprehensive quantitative comparison of our method against 20 state-of-the-art approaches on the EORSSD, ORSSD, and ORSI4199 datasets. On the ORSSD dataset, our proposed method achieved superior overall performance, ranking first in five evaluation metrics and second in the remaining metric. Notably, PA-KRN previously demonstrated the highest performance among NSI-SOD methods. However, our method outperforms PA-KRN, with significant improvements of 2.52%, 1.80%, 1.60%, 3.93%, and 3.51% in $[eqn]$ , $[eqn]$ , $[eqn]$ , $[eqn]$ , and $[eqn]$ , respectively. While our approach shows slightly lower performance in the S-measure metric compared to the leading ORSI-SOD method, LSHNet, it achieves notable enhancements in all other five metrics, particularly improvements of 0.30% and 0.18% in $[eqn]$ and $[eqn]$ , respectively.

On the larger EORSSD dataset, our method also achieved outstanding performance. Compared with the PA-KRN, our approach demonstrated improvements of 2.45%, 4.24%, 1.59%, 2.10%, and 2.00% in terms of $[eqn]$ , $[eqn]$ , $[eqn]$ , $[eqn]$ , and $[eqn]$ , respectively. When compared to the second-best method, LSHNet, our method demonstrated improvements of 0.42% and 0.80% in terms of $[eqn]$ and $[eqn]$ , respectively. Moreover, we observed a substantial improvement in MAE, achieving a notable reduction of 9.38% compared to LSHNet, demonstrating the superiority of our proposed method.

On the challenging ORSI4199 dataset, our method exhibited state-of-the-art performance, achieving the highest rank across all six evaluation metrics. Specifically, our model substantially surpassed PA-KRN by 4.03%, 3.04%, 2.45%, 5.30%, and 3.89% in terms of $[eqn]$ , $[eqn]$ , $[eqn]$ , $[eqn]$ , and $[eqn]$ , respectively. Compared to the second-best model, our approach improved critical metrics of $[eqn]$ and $[eqn]$ by 3.0% and 0.10%, respectively. Figure 4 illustrates the PR curves on the three benchmark datasets, demonstrating that our MCFNet achieves superior precision and recall. Notably, the PR curves for our method are positioned closest to the upper-right corner, indicating consistently higher performance compared with all of the other evaluated methods.

Overall quantitative analyses confirm that our proposed model achieves new state-of-the-art performance on the ORSI-SOD task, striking a favorable balance between accuracy and computational efficiency. Furthermore, a comparative evaluation between specialized ORSI-SOD and general NSI-SOD methods reveals a significant advantage of ORSI-specific models. These findings underscore the critical importance and urgency of developing dedicated approaches tailored explicitly for remote sensing imagery.

(3) Qualitative Analysis. In Figure 5, we present representative and challenging ORSI examples to qualitatively compare MCFNet with nine state-of-the-art approaches. These scenarios encompass cases such as interference from surrounding elements, clusters of small objects, shadow-covered targets, complex background textures, low-contrast regions, and irregularly shaped structures. In remote sensing imagery, non-salient regions frequently cause distractions, which can result in inaccurate detection results. As illustrated in the first and second rows of Figure 5, other methods such as LSHNet, CorrNet, and EMFINet exhibit limitations, either fragmenting target shapes or misidentifying background objects as salient targets. In contrast, our proposed method effectively addresses these challenges by preserving the integrity of salient objects while mitigating interference. MCFNet accurately differentiates salient targets from distractors by leveraging contextual modeling. Furthermore, the integration of SAM guides the model to focus on discriminative features relevant to the salient target while suppressing the influence of non-salient objects. This design enables our model to robustly isolate and highlight salient objects even in complex scenes with potential interferences.

In ORSI, the variability in shooting distances and angles often results in objects exhibiting tiny features, particularly in scenes containing multiple small targets, which presents a significant challenge for SOD. The third and fourth rows of Figure 5 illustrate qualitative results in such complex scenarios. As evident from the visualizations, existing state-of-the-art methods (i.e., LSHNet, ADSTNet, and GateNet) often lead to omissions. Meanwhile, methods such as SFANet and PA-KRN erroneously identify non-target regions as salient objects. In contrast, our method demonstrates superior performance in multiple tiny object scenarios, accurately detecting all targets without false positives or omissions.

Shadows present a significant challenge in SOD, as they are often misclassified as salient regions, leading to inaccurate detection. As illustrated in the fifth and sixth rows of Figure 5, many existing methods erroneously highlight shadowed areas, which affects the completeness of target detection. In contrast, our proposed approach accurately identifies target objects while preserving their structural integrity, effectively mitigating the interference of redundant shadow regions.

Effectively mitigating background interference is a critical challenge in salient object detection, particularly in complex environments. As shown in row 7 of Figure 5, all comparative methods, except for our proposed MCFNet, fail to suppress background interference, resulting in excessive background inclusion and incomplete object detection. In row 8, only our method successfully isolates the salient target from the background, achieving precise and robust detection. These results strongly demonstrate the effectiveness and resilience of our model in handling complex background conditions.

Low-contrast object detection remains a challenging aspect of SOD, as salient targets often exhibit visual similarity to the background, making them difficult to distinguish. As shown in the low-contrast bridge detection example in row 9 of Figure 5, existing methods struggle to accurately delineate the complete structure of the bridge, resulting in incomplete predictions. Similarly, in the scenario presented in row 10, methods such as ADSTNet, EMFINet, and PA-KRN fail to correctly identify the true target object, mistakenly segmenting visually similar background regions instead. In contrast, our method not only accurately detects the target but also precisely delineates its boundaries, demonstrating superior robustness in low-contrast settings.

Salient objects in remote sensing images often exhibit complex and irregular topologies, making accurate edge delineation particularly challenging. In practice, methods such as LSHNet, SFANet, and MJRBM frequently fail to capture the full extent of rivers and buildings, producing fragmented or jagged boundaries that poorly represent the true geometry of the targets. In contrast, our method demonstrates superior performance by accurately detecting rivers with well-defined contours and preserving the structural integrity of buildings. This advantage stems from the integration of a semantic-aware attention mechanism and an adjacent contextual information fusion strategy. The semantic-aware attention mechanism enhances detection completeness by guiding the model to focus more holistically on key target features. Simultaneously, the contextual interaction strategy significantly improves boundary sharpness and enables the model to more effectively handle irregular geometric structures.

4.3. Ablation Experiment

In this study, we conducted a series of ablation experiments to systematically evaluate the effectiveness of each component within MCFNet. Specifically, our analysis focused on four key aspects: (1) the individual contribution of each module, (2) the effectiveness of branches in CIM, (3) the effectiveness of semantic-aware attention, and (4) the model’s sensitivity to different backbone architectures. To ensure experimental fairness, each ablation variant was retrained under identical configurations, and their performance was rigorously evaluated and compared.

Effectiveness of the proposed module. To assess the practical contribution of the proposed modules, we designed four model variants for ablation studies. Variant 1 serves as the baseline model, incorporating the Res2Net backbone and the decoder. Variant 2 integrates SAM into the baseline, while variant 3 includes the CIM. Variant 4 combines both SAM and CIM, representing the full version of our model. Experiments were conducted on the ORSSD and EORSSD datasets, with quantitative results summarized in Table 2. With the integration of SAM, variant 2 achieved notable performance gains on the ORSSD dataset, with improvements of 1.51%, 0.50%, 0.46%, and 0.0019 in terms of $[eqn]$ , $[eqn]$ , $[eqn]$ , and $[eqn]$ , respectively. With the addition of SAM, the $[eqn]$ improved by 0.67% and 1.21% on the EORSSD and ORSI4199 datasets, respectively. Similarly, the addition of the CIM results in consistent and significant enhancements across all evaluation metrics. When SAM and CIM are incorporated, the model demonstrates the most substantial performance improvements in $[eqn]$ , $[eqn]$ , $[eqn]$ , and $[eqn]$ surpassing the baseline by 1.91%, 0.96%, 1.91%, and 0.0039 on the EORSSD dataset, respectively. These results indicate that each module plays an indispensable role, as removing any one of them results in a noticeable decline in performance. As illustrated in the visual results in Figure 6, the incorporation of SAM can accurately strengthen the localization ability of objects, allowing it to more accurately identify salient regions and suppress background interference. Meanwhile, CIM facilitates effective fusion and interaction between high-level semantic features and low-level texture details, leading to a more coherent and natural delineation of object contours.

The effectiveness of branches in CIM. To further assess the contribution of individual branches within the CIM module, we designed two ablated variants: one excluding the adjacent branches (w/o AB) and the other omitting the current branch (w/o CB). As shown in Table 3, both variants exhibit noticeable drops in performance relative to the full CIM configuration. These results underscore the importance of both branch types in boosting the overall effectiveness of the model. These findings confirm the effectiveness of the dual-branch design in facilitating cross-scale information fusion and underscore the importance of incorporating both current and neighboring contextual features. Specifically, removal of the current branch leads to a substantial performance drop on the ORSSD dataset, with declines of 1.03%, 0.68%, and 1.05% in $[eqn]$ , $[eqn]$ , and $[eqn]$ on ORSSD, respectively. A similar trend is observed on the EORSSD dataset, where w/o CB causes substantial drops of 1.40%, 0.89%, and 0.87% in $[eqn]$ , $[eqn]$ , and $[eqn]$ , respectively. The performance on the ORSI4199 dataset drops significantly when either the current branch or adjacent branches are removed, with decreases of 1.08% and 0.67% in $[eqn]$ , and 1.20% and 1.05% in $[eqn]$ , respectively. Although the removal of the adjacent branches also leads to a measurable decline, its effect is less pronounced, further highlighting the critical role of the current branch in maintaining performance.

The effectiveness of semantic-aware attention. To further assess the effectiveness of semantic-aware attention (SAM), we conducted two variant experiments by replacing semantic-aware attention with either self-attention [52] or gated attention [53]. As shown in Figure 7, the corresponding heat maps offer an intuitive visualization of their respective performances. In the self-attention variant, we observe a more uniform attention distribution across the feature map, reflecting a stronger global capacity for capturing overall contextual information. However, compared with semantic-aware attention, self-attention is less adept at extracting fine-grained features in regions requiring precise semantic cues. This limitation is particularly evident when objects share similar appearances, where self-attention often struggles to differentiate foreground from background, leading to reduced performance. In contrast, the gated attention variant concentrates attention more narrowly through a gating mechanism, thereby highlighting critical features and improving feature selection accuracy. Despite this advantage, gated attention lacks the deeper semantic comprehension inherent to semantic-aware attention. Consequently, when dealing with optical remote sensing images containing complex backgrounds, gated attention underperforms due to its more superficial handling of implicit semantic relationships.

The sensitivity to various backbones. In evaluating our module’s sensitivity to different backbone networks, we have chosen three well-known encoders, VGG16, PVTv2, and ResNet-50. For each encoder, we constructed a corresponding variant model (MCFNet-VGG, MCFNet-PVT, and MCFNet-ResNet) that preserves our model’s core structure while substituting the backbone, facilitating a comprehensive assessment of performance. Table 4 provides detailed metrics for each variant. Across all three backbones, our proposed model always has a stable performance, which demonstrates robustness and flexibility. Further analysis revealed that the Res2Net backbone yielded superior results. Consequently, Res2Net was chosen as the default backbone for the final model configuration.

5. Conclusions

ORSI-SOD is notably challenging due to the diversity in target number and scale, complex background environments, and irregular topologies. To tackle these issues, we proposed a novel Multi-scale Contextual Fusion Network (MCFNet), comprising a Semantic-Aware Attention Module (SAM) and a Contextual Interconnection Module (CIM). Specifically, SAM introduces preliminary semantic masks to provide accurate semantic guidance during feature extraction, enabling the model to capture long-range dependencies and localize targets robustly. Meanwhile, CIM fuses contextual information across adjacent feature levels through multi-scale refinement, thereby enriching and optimizing texture representations at different scales. This cross-level interaction not only bolsters the model’s ability to discern salient target features but also improves its capacity to suppress background interference, thus increasing robustness. Extensive evaluations and ablation studies on three widely used datasets demonstrate that MCFNet surpasses 20 competing methods by producing more complete and accurate salient regions while maintaining robust detection performance even in complex scenarios.