Fusformer: A Transformer-based Fusion Approach for Hyperspectral Image Super-resolution

TL;DR

This paper introduces Fusformer, a transformer-based model that effectively fuses low-resolution hyperspectral images with high-resolution multispectral images to produce high-resolution hyperspectral images, overcoming local limitations of CNNs.

Contribution

The paper proposes a novel transformer-based fusion network for hyperspectral super-resolution, emphasizing global feature exploration and spectral structure preservation.

Findings

Outperforms state-of-the-art methods in quality metrics

Effectively captures global feature relationships

Reduces mapping complexity for better performance

Abstract

Hyperspectral image has become increasingly crucial due to its abundant spectral information. However, It has poor spatial resolution with the limitation of the current imaging mechanism. Nowadays, many convolutional neural networks have been proposed for the hyperspectral image super-resolution problem. However, convolutional neural network (CNN) based methods only consider the local information instead of the global one with the limited kernel size of receptive field in the convolution operation. In this paper, we design a network based on the transformer for fusing the low-resolution hyperspectral images and high-resolution multispectral images to obtain the high-resolution hyperspectral images. Thanks to the representing ability of the transformer, our approach is able to explore the intrinsic relationships of features globally. Furthermore, considering the LR-HSIs hold the main spectral structure, the network focuses on the spatial detail estimation releasing from the burden of reconstructing the whole data. It reduces the mapping space of the proposed network, which enhances the final performance. Various experiments and quality indexes show our approach's superiority compared with other state-of-the-art methods.