MFmamba: A Multi-function Network for Panchromatic Image Resolution Restoration Based on State-Space Model

TL;DR

MFmamba is a novel multi-function neural network that simultaneously performs super-resolution, spectral recovery, and their combination on panchromatic images, effectively enhancing spatial and spectral resolution with a unified model.

Contribution

The paper introduces MFmamba, a multi-function network based on a state-space model that integrates super-resolution and spectral recovery tasks for panchromatic images.

Findings

MFmamba achieves competitive evaluation metrics.

The model performs well in visual results.

It effectively handles three tasks with only a PAN image input.

Abstract

Remote sensing images are becoming increasingly widespread in military, earth resource exploration. Because of the limitation of a single sensor, we can obtain high spatial resolution grayscale panchromatic (PAN) images and low spatial resolution color multispectral (MS) images. Therefore, an important issue is to obtain a color image with high spatial resolution when there is only a PAN image at the input. The existing methods improve spatial resolution using super-resolution (SR) technology and spectral recovery using colorization technology. However, the SR technique cannot improve the spectral resolution, and the colorization technique cannot improve the spatial resolution. Moreover, the pansharpening method needs two registered inputs and can not achieve SR. As a result, an integrated approach is expected. To solve the above problems, we designed a novel multi-function model (MFmamba) to realize the tasks of SR, spectral recovery, joint SR and spectral recovery through three different inputs. Firstly, MFmamba utilizes UNet++ as the backbone, and a Mamba Upsample Block (MUB) is combined with UNet++. Secondly, a Dual Pool Attention (DPA) is designed to replace the skip connection in UNet++. Finally, a Multi-scale Hybrid Cross Block (MHCB) is proposed for initial feature extraction. Many experiments show that MFmamba is competitive in evaluation metrics and visual results and performs well in the three tasks when only the input PAN image is used.