SpectraMorph: Structured Latent Learning for Self-Supervised Hyperspectral Super-Resolution

TL;DR

SpectraMorph is a physics-guided, self-supervised hyperspectral super-resolution framework that uses a structured latent space and unmixing bottleneck to produce interpretable results and outperform existing methods.

Contribution

It introduces a novel structured latent learning approach with an unmixing bottleneck, improving interpretability and robustness in hyperspectral super-resolution.

Findings

Outperforms state-of-the-art unsupervised/self-supervised methods.

Remains effective with very few MSI bands, including single-band pan-chromatic images.

Trains in under a minute, demonstrating efficiency.

Abstract

Hyperspectral sensors capture dense spectra per pixel but suffer from low spatial resolution, causing blurred boundaries and mixed-pixel effects. Co-registered companion sensors such as multispectral, RGB, or panchromatic cameras provide high-resolution spatial detail, motivating hyperspectral super-resolution through the fusion of hyperspectral and multispectral images (HSI-MSI). Existing deep learning based methods achieve strong performance but rely on opaque regressors that lack interpretability and often fail when the MSI has very few bands. We propose SpectraMorph, a physics-guided self-supervised fusion framework with a structured latent space. Instead of direct regression, SpectraMorph enforces an unmixing bottleneck: endmember signatures are extracted from the low-resolution HSI, and a compact multilayer perceptron predicts abundance-like maps from the MSI. Spectra are reconstructed by linear mixing, with training performed in a self-supervised manner via the MSI sensor's spectral response function. SpectraMorph produces interpretable intermediates, trains in under a minute, and remains robust even with a single-band (pan-chromatic) MSI. Experiments on synthetic and real-world datasets show SpectraMorph consistently outperforming state-of-the-art unsupervised/self-supervised baselines while remaining very competitive against supervised baselines.