RAPNet: A Receptive-Field Adaptive Convolutional Neural Network for Pansharpening

TL;DR

RAPNet introduces content-adaptive convolution and an attention-based fusion module to improve pansharpening, achieving higher spatial detail and spectral fidelity in remote sensing images.

Contribution

The paper proposes RAPNet, a novel CNN architecture with receptive-field adaptive convolution and dynamic feature fusion for enhanced pansharpening.

Findings

Outperforms existing methods in quantitative metrics

Provides better spatial detail and spectral fidelity

Ablation studies confirm effectiveness of adaptive components

Abstract

Pansharpening refers to the process of integrating a high resolution panchromatic (PAN) image with a lower resolution multispectral (MS) image to generate a fused product, which is pivotal in remote sensing. Despite the effectiveness of CNNs in addressing this challenge, they are inherently constrained by the uniform application of convolutional kernels across all spatial positions, overlooking local content variations. To overcome this issue, we introduce RAPNet, a new architecture that leverages content-adaptive convolution. At its core, RAPNet employs the Receptive-field Adaptive Pansharpening Convolution (RAPConv), designed to produce spatially adaptive kernels responsive to local feature context, thereby enhancing the precision of spatial detail extraction. Additionally, the network integrates the Pansharpening Dynamic Feature Fusion (PAN-DFF) module, which incorporates an attention mechanism to achieve an optimal balance between spatial detail enhancement and spectral fidelity. Comprehensive evaluations on publicly available datasets confirm that RAPNet delivers superior performance compared to existing approaches, as demonstrated by both quantitative metrics and qualitative assessments. Ablation analyses further substantiate the effectiveness of the proposed adaptive components.