Cross-Scale Pansharpening via ScaleFormer and the PanScale Benchmark

TL;DR

This paper introduces a new large-scale dataset and benchmark for cross-scale pansharpening, along with a novel ScaleFormer architecture that improves generalization across different image resolutions and scales.

Contribution

The paper presents ScaleFormer, a new architecture for multi-scale pansharpening, and the PanScale dataset with PanScale-Bench for evaluating cross-scale generalization.

Findings

ScaleFormer outperforms state-of-the-art methods in fusion quality.

The approach demonstrates superior cross-scale generalization.

Extensive experiments validate the effectiveness of the proposed method.

Abstract

Pansharpening aims to generate high-resolution multi-spectral images by fusing the spatial detail of panchromatic images with the spectral richness of low-resolution MS data. However, most existing methods are evaluated under limited, low-resolution settings, limiting their generalization to real-world, high-resolution scenarios. To bridge this gap, we systematically investigate the data, algorithmic, and computational challenges of cross-scale pansharpening. We first introduce PanScale, the first large-scale, cross-scale pansharpening dataset, accompanied by PanScale-Bench, a comprehensive benchmark for evaluating generalization across varying resolutions and scales. To realize scale generalization, we propose ScaleFormer, a novel architecture designed for multi-scale pansharpening. ScaleFormer reframes generalization across image resolutions as generalization across sequence lengths: it tokenizes images into patch sequences of the same resolution but variable length proportional to image scale. A Scale-Aware Patchify module enables training for such variations from fixed-size crops. ScaleFormer then decouples intra-patch spatial feature learning from inter-patch sequential dependency modeling, incorporating Rotary Positional Encoding to enhance extrapolation to unseen scales. Extensive experiments show that our approach outperforms SOTA methods in fusion quality and cross-scale generalization. The datasets and source code are available at https://github.com/caoke-963/ScaleFormer.