Proximal PanNet: A Model-Based Deep Network for Pansharpening

TL;DR

Proximal PanNet introduces a model-based deep learning approach for pansharpening by unfolding an iterative algorithm into a neural network, enhancing interpretability and performance in generating high-resolution multispectral images.

Contribution

This work combines convolutional sparse coding with deep learning by unfolding an iterative algorithm into a neural network for improved pansharpening.

Findings

Outperforms existing methods quantitatively.

Achieves better qualitative results.

End-to-end learnable framework.

Abstract

Recently, deep learning techniques have been extensively studied for pansharpening, which aims to generate a high resolution multispectral (HRMS) image by fusing a low resolution multispectral (LRMS) image with a high resolution panchromatic (PAN) image. However, existing deep learning-based pansharpening methods directly learn the mapping from LRMS and PAN to HRMS. These network architectures always lack sufficient interpretability, which limits further performance improvements. To alleviate this issue, we propose a novel deep network for pansharpening by combining the model-based methodology with the deep learning method. Firstly, we build an observation model for pansharpening using the convolutional sparse coding (CSC) technique and design a proximal gradient algorithm to solve this model. Secondly, we unfold the iterative algorithm into a deep network, dubbed as Proximal PanNet, by learning the proximal operators using convolutional neural networks. Finally, all the learnable modules can be automatically learned in an end-to-end manner. Experimental results on some benchmark datasets show that our network performs better than other advanced methods both quantitatively and qualitatively.