CGFformer: Cluster-Guidance Frequency Transformer for Pansharpening

TL;DR

CGFformer is a novel Transformer-based model for pansharpening that adaptively separates and fuses frequency components, leading to improved high-resolution multispectral image reconstruction.

Contribution

The paper introduces an adaptive separation module and a dual-stream refinement approach that better exploit frequency information for enhanced pansharpening performance.

Findings

CGFformer outperforms existing methods on multiple benchmarks.

The adaptive separation improves frequency component discrimination.

The dual-stream module effectively suppresses various noise types.

Abstract

Pansharpening aims to generate high-resolution multispectral (HRMS) images by fusing low-resolution multispectral (LRMS) images with high-resolution panchromatic (PAN) images. However, the current mainstream frequency-based pansharpening methods employ fixed frequency filters, which cannot precisely adapt to complex and spatially diversified frequency distributions in PAN and MS images. Furthermore, existing denoising strategies insufficiently exploit frequency components for denoising and struggle to suppress various noise types accurately. To address these challenges, we propose CGFformer, a cluster-guidance frequency Transformer that focuses on varying frequency distribution and interactions between frequency and spatial components. Specifically, we design an adaptive separation module that integrates local features and non-local information through K-means clustering, enabling more precise separation of high- and low-frequency components. Subsequently, we introduce a dual-stream refinement module combined with Transformer-based cross-attention to remove various noise, allowing the network to jointly suppress frequency-relevant and irrelevant disturbances. In addition, we develop a frequency-spatial fusion module designed to enhance detail and facilitate spatial-frequency interaction, ensuring more effective reconstruction of spatial structures in the fused results. Extensive experiments on multiple benchmark datasets demonstrate that the proposed CGFformer achieves notable improvements over existing pansharpening approaches.