Unsupervised Hyperspectral Image Super-Resolution via Self-Supervised Modality Decoupling

TL;DR

This paper introduces an unsupervised hyperspectral image super-resolution method called MossFuse that decouples shared and unique modality information using self-supervision, leading to improved fusion quality and efficiency.

Contribution

The paper proposes a novel self-supervised framework that effectively decouples shared and complementary features across modalities for hyperspectral image super-resolution.

Findings

Outperforms existing methods on multiple datasets.

Requires fewer parameters and less inference time.

Demonstrates the effectiveness of modality decoupling with subspace clustering loss.

Abstract

Fusion-based hyperspectral image super-resolution aims to fuse low-resolution hyperspectral images (LR-HSIs) and high-resolution multispectral images (HR-MSIs) to reconstruct high spatial and high spectral resolution images. Current methods typically apply direct fusion from the two modalities without effective supervision, leading to an incomplete perception of deep modality-complementary information and a limited understanding of inter-modality correlations. To address these issues, we propose a simple yet effective solution for unsupervised HMIF, revealing that modality decoupling is key to improving fusion performance. Specifically, we propose an end-to-end self-supervised Modality-Decoupled Spatial-Spectral Fusion (MossFuse) framework that decouples shared and complementary information across modalities and aggregates a concise representation of both LR-HSIs and HR-MSIs to reduce modality redundancy. Also, we introduce the subspace clustering loss as a clear guide to decouple modality-shared features from modality-complementary ones. Systematic experiments over multiple datasets demonstrate that our simple and effective approach consistently outperforms the existing HMIF methods while requiring considerably fewer parameters with reduced inference time. The source source code is in \href{https://github.com/dusongcheng/MossFuse}{MossFuse}.