LDP-Net: An Unsupervised Pansharpening Network Based on Learnable Degradation Processes

TL;DR

LDP-Net is an unsupervised pansharpening network that learns degradation processes to fuse low-resolution multispectral and panchromatic images effectively without requiring high-resolution multispectral training data.

Contribution

The paper introduces a novel unsupervised deep learning approach with learnable degradation modules and a hybrid loss for improved pansharpening performance.

Findings

Effective fusion of PAN and LRMS images demonstrated on Worldview datasets.

Achieves competitive results without supervised HRMS data.

Maintains spectral and spatial consistency in fused images.

Abstract

Pansharpening in remote sensing image aims at acquiring a high-resolution multispectral (HRMS) image directly by fusing a low-resolution multispectral (LRMS) image with a panchromatic (PAN) image. The main concern is how to effectively combine the rich spectral information of LRMS image with the abundant spatial information of PAN image. Recently, many methods based on deep learning have been proposed for the pansharpening task. However, these methods usually has two main drawbacks: 1) requiring HRMS for supervised learning; and 2) simply ignoring the latent relation between the MS and PAN image and fusing them directly. To solve these problems, we propose a novel unsupervised network based on learnable degradation processes, dubbed as LDP-Net. A reblurring block and a graying block are designed to learn the corresponding degradation processes, respectively. In addition, a novel hybrid loss function is proposed to constrain both spatial and spectral consistency between the pansharpened image and the PAN and LRMS images at different resolutions. Experiments on Worldview2 and Worldview3 images demonstrate that our proposed LDP-Net can fuse PAN and LRMS images effectively without the help of HRMS samples, achieving promising performance in terms of both qualitative visual effects and quantitative metrics.