Hyperspectral Image Super-resolution with Deep Priors and Degradation Model Inversion

TL;DR

This paper introduces a hyperspectral image super-resolution method that integrates a physical degradation model with deep learning priors, leading to improved fusion of low-resolution hyperspectral and high-resolution RGB images.

Contribution

It combines a physical degradation model with deep neural network-based priors for enhanced hyperspectral image super-resolution, addressing limitations of previous deep learning approaches.

Findings

Performance improved with the proposed method

Effective fusion of LR hyperspectral and HR RGB images

Deep prior regularization enhances super-resolution quality

Abstract

To overcome inherent hardware limitations of hyperspectral imaging systems with respect to their spatial resolution, fusion-based hyperspectral image (HSI) super-resolution is attracting increasing attention. This technique aims to fuse a low-resolution (LR) HSI and a conventional high-resolution (HR) RGB image in order to obtain an HR HSI. Recently, deep learning architectures have been used to address the HSI super-resolution problem and have achieved remarkable performance. However, they ignore the degradation model even though this model has a clear physical interpretation and may contribute to improve the performance. We address this problem by proposing a method that, on the one hand, makes use of the linear degradation model in the data-fidelity term of the objective function and, on the other hand, utilizes the output of a convolutional neural network for designing a deep prior regularizer in spectral and spatial gradient domains. Experiments show the performance improvement achieved with this strategy.