Deep Posterior Distribution-based Embedding for Hyperspectral Image Super-resolution

TL;DR

This paper introduces PDE-Net, a deep learning framework for hyperspectral image super-resolution that models high-dimensional embedding as a posterior distribution, leading to improved performance and uncertainty estimation.

Contribution

It formulates hyperspectral embedding as a posterior distribution approximation and integrates it into a physically interpretable super-resolution network, PDE-Net.

Findings

PDE-Net outperforms state-of-the-art methods on benchmark datasets.

The probabilistic embedding provides epistemic uncertainty estimates.

The method is lightweight and physically interpretable.

Abstract

In this paper, we investigate the problem of hyperspectral (HS) image spatial super-resolution via deep learning. Particularly, we focus on how to embed the high-dimensional spatial-spectral information of HS images efficiently and effectively. Specifically, in contrast to existing methods adopting empirically-designed network modules, we formulate HS embedding as an approximation of the posterior distribution of a set of carefully-defined HS embedding events, including layer-wise spatial-spectral feature extraction and network-level feature aggregation. Then, we incorporate the proposed feature embedding scheme into a source-consistent super-resolution framework that is physically-interpretable, producing lightweight PDE-Net, in which high-resolution (HR) HS images are iteratively refined from the residuals between input low-resolution (LR) HS images and pseudo-LR-HS images degenerated from reconstructed HR-HS images via probability-inspired HS embedding. Extensive experiments over three common benchmark datasets demonstrate that PDE-Net achieves superior performance over state-of-the-art methods. Besides, the probabilistic characteristic of this kind of networks can provide the epistemic uncertainty of the network outputs, which may bring additional benefits when used for other HS image-based applications. The code will be publicly available at https://github.com/jinnh/PDE-Net.