DSConv: Dynamic Splitting Convolution for Pansharpening

TL;DR

This paper introduces DSConv, a dynamic splitting convolution method with attention for pansharpening, improving feature extraction and achieving state-of-the-art results in remote sensing image fusion.

Contribution

It proposes a novel dynamic kernel splitting strategy with attention, enhancing feature extraction and network performance for pansharpening tasks.

Findings

Achieves state-of-the-art performance on pansharpening benchmarks.

Demonstrates improved feature extraction and generalization.

Outperforms existing convolution methods in remote sensing image fusion.

Abstract

Aiming to obtain a high-resolution image, pansharpening involves the fusion of a multi-spectral image (MS) and a panchromatic image (PAN), the low-level vision task remaining significant and challenging in contemporary research. Most existing approaches rely predominantly on standard convolutions, few making the effort to adaptive convolutions, which are effective owing to the inter-pixel correlations of remote sensing images. In this paper, we propose a novel strategy for dynamically splitting convolution kernels in conjunction with attention, selecting positions of interest, and splitting the original convolution kernel into multiple smaller kernels, named DSConv. The proposed DSConv more effectively extracts features of different positions within the receptive field, enhancing the network's generalization, optimization, and feature representation capabilities. Furthermore, we innovate and enrich concepts of dynamic splitting convolution and provide a novel network architecture for pansharpening capable of achieving the tasks more efficiently, building upon this methodology. Adequate fair experiments illustrate the effectiveness and the state-of-the-art performance attained by DSConv.Comprehensive and rigorous discussions proved the superiority and optimal usage conditions of DSConv.