PC-GANs: Progressive Compensation Generative Adversarial Networks for Pan-sharpening

TL;DR

This paper introduces a novel two-step progressive GAN-based model for pan-sharpening that enhances multispectral images by sequentially refining spatial and spectral details, outperforming traditional one-step methods.

Contribution

The paper proposes a triple GAN architecture with a joint loss function for progressive spatial and spectral compensation in pan-sharpening, improving detail preservation and fusion quality.

Findings

Outperforms existing methods on multiple datasets

Effectively preserves spectral and spatial details

Demonstrates efficiency and robustness in experiments

Abstract

The fusion of multispectral and panchromatic images is always dubbed pansharpening. Most of the available deep learning-based pan-sharpening methods sharpen the multispectral images through a one-step scheme, which strongly depends on the reconstruction ability of the network. However, remote sensing images always have large variations, as a result, these one-step methods are vulnerable to the error accumulation and thus incapable of preserving spatial details as well as the spectral information. In this paper, we propose a novel two-step model for pan-sharpening that sharpens the MS image through the progressive compensation of the spatial and spectral information. Firstly, a deep multiscale guided generative adversarial network is used to preliminarily enhance the spatial resolution of the MS image. Starting from the pre-sharpened MS image in the coarse domain, our approach then progressively refines the spatial and spectral residuals over a couple of generative adversarial networks (GANs) that have reverse architectures. The whole model is composed of triple GANs, and based on the specific architecture, a joint compensation loss function is designed to enable the triple GANs to be trained simultaneously. Moreover, the spatial-spectral residual compensation structure proposed in this paper can be extended to other pan-sharpening methods to further enhance their fusion results. Extensive experiments are performed on different datasets and the results demonstrate the effectiveness and efficiency of our proposed method.