Fast Model-guided Instance-wise Adaptation Framework for Real-world Pansharpening with Fidelity Constraints

TL;DR

This paper introduces FMG-Pan, a fast, generalizable, model-guided instance-wise adaptation framework for real-world pansharpening that achieves high-quality fusion with rapid training and inference.

Contribution

The proposed FMG-Pan framework combines a pretrained model with a lightweight adaptive network, enabling fast, cross-sensor pansharpening with fidelity constraints and superior performance.

Findings

FMG-Pan achieves state-of-the-art results on real-world datasets.

It completes training and inference within 3 seconds for a 512x512x8 image.

The method outperforms existing zero-shot approaches in speed and quality.

Abstract

Pansharpening aims to generate high-resolution multispectral (HRMS) images by fusing low-resolution multispectral (LRMS) and high-resolution panchromatic (PAN) images while preserving both spectral and spatial information. Although deep learning (DL)-based pansharpening methods achieve impressive performance, they require high training cost and large datasets, and often degrade when the test distribution differs from training, limiting generalization. Recent zero-shot methods, trained on a single PAN/LRMS pair, offer strong generalization but suffer from limited fusion quality, high computational overhead, and slow convergence. To address these issues, we propose FMG-Pan, a fast and generalizable model-guided instance-wise adaptation framework for real-world pansharpening, achieving both cross-sensor generality and rapid training-inference. The framework leverages a pretrained model to guide a lightweight adaptive network through joint optimization with spectral and physical fidelity constraints. We further design a novel physical fidelity term to enhance spatial detail preservation. Extensive experiments on real-world datasets under both intra- and cross-sensor settings demonstrate state-of-the-art performance. On the WorldView-3 dataset, FMG-Pan completes training and inference for a 512x512x8 image within 3 seconds on an RTX 3090 GPU, significantly faster than existing zero-shot methods, making it suitable for practical deployment.