Two-Stage Random Alternation Framework for One-Shot Pansharpening

TL;DR

This paper introduces TRA-PAN, a two-stage framework that performs instance-specific optimization for pansharpening, significantly improving generalization and fusion quality on unseen real-world image pairs.

Contribution

The paper proposes a novel two-stage random alternation framework that enables one-shot, instance-specific pansharpening, addressing generalization issues of traditional deep learning models.

Findings

TRA-PAN outperforms SOTA methods in quantitative metrics.

It achieves superior visual quality in real-world scenarios.

The framework enhances robustness and practical applicability.

Abstract

Deep learning has substantially advanced pansharpening, achieving impressive fusion quality. However, a prevalent limitation is that conventional deep learning models, which typically rely on training datasets, often exhibit suboptimal generalization to unseen real-world image pairs. This restricts their practical utility when faced with real-world scenarios not included in the training datasets. To overcome this, we introduce a two-stage random alternating framework (TRA-PAN) that performs instance-specific optimization for any given Multispectral(MS)/Panchromatic(PAN) pair, ensuring robust and high-quality fusion. TRA-PAN effectively integrates strong supervision constraints from reduced-resolution images with the physical characteristics of the full-resolution images. The first stage introduces a pre-training procedure, which includes Degradation-Aware Modeling (DAM) to capture spectral degradation mappings, alongside a warm-up procedure designed to reduce training time and mitigate the adverse effects of reduced-resolution data. The second stage employs Random Alternation Optimization (RAO), randomly alternating between reduced- and full-resolution images to refine the fusion model progressively. This adaptive, per-instance optimization strategy, operating in a one-shot manner for each MS/PAN pair, yields superior high-resolution multispectral images. Experimental results demonstrate that TRA-PAN outperforms state-of-the-art (SOTA) methods in quantitative metrics and visual quality in real-world scenarios, underscoring its enhanced practical applicability and robustness.