Multispectral to Hyperspectral using Pretrained Foundational model

TL;DR

This paper introduces pretrained transformer models that reconstruct hyperspectral images from multispectral data, aiming to improve atmospheric monitoring by leveraging the strengths of both imaging types.

Contribution

It presents novel spectral and spatial-spectral transformer models pretrained on large datasets and fine-tuned on aligned image pairs for hyperspectral reconstruction from multispectral inputs.

Findings

Models successfully reconstruct hyperspectral data from multispectral images.

Pretraining on large datasets improves reconstruction accuracy.

Potential to enhance greenhouse gas monitoring capabilities.

Abstract

Hyperspectral imaging provides detailed spectral information, offering significant potential for monitoring greenhouse gases like CH4 and NO2. However, its application is constrained by limited spatial coverage and infrequent revisit times. In contrast, multispectral imaging delivers broader spatial and temporal coverage but lacks the spectral granularity required for precise GHG detection. To address these challenges, this study proposes Spectral and Spatial-Spectral transformer models that reconstruct hyperspectral data from multispectral inputs. The models in this paper are pretrained on EnMAP and EMIT datasets and fine-tuned on spatio-temporally aligned (Sentinel-2, EnMAP) and (HLS-S30, EMIT) image pairs respectively. Our model has the potential to enhance atmospheric monitoring by combining the strengths of hyperspectral and multispectral imaging systems.