Adaptive Rectangular Convolution for Remote Sensing Pansharpening

TL;DR

This paper introduces ARConv, an adaptive convolutional module that dynamically adjusts kernel size and sampling points to better capture multi-scale features in remote sensing images, significantly improving pansharpening results.

Contribution

The paper proposes ARConv, a novel adaptive convolutional module that learns optimal kernel dimensions and sampling locations, addressing fixed sampling limitations in traditional CNNs for remote sensing.

Findings

ARNet with ARConv outperforms existing methods in multiple datasets.

Ablation studies confirm the effectiveness of adaptive kernel adjustments.

Visualization demonstrates improved feature extraction with ARConv.

Abstract

Recent advancements in convolutional neural network (CNN)-based techniques for remote sensing pansharpening have markedly enhanced image quality. However, conventional convolutional modules in these methods have two critical drawbacks. First, the sampling positions in convolution operations are confined to a fixed square window. Second, the number of sampling points is preset and remains unchanged. Given the diverse object sizes in remote sensing images, these rigid parameters lead to suboptimal feature extraction. To overcome these limitations, we introduce an innovative convolutional module, Adaptive Rectangular Convolution (ARConv). ARConv adaptively learns both the height and width of the convolutional kernel and dynamically adjusts the number of sampling points based on the learned scale. This approach enables ARConv to effectively capture scale-specific features of various objects within an image, optimizing kernel sizes and sampling locations. Additionally, we propose ARNet, a network architecture in which ARConv is the primary convolutional module. Extensive evaluations across multiple datasets reveal the superiority of our method in enhancing pansharpening performance over previous techniques. Ablation studies and visualization further confirm the efficacy of ARConv.