Unsupervised Spectral Demosaicing with Lightweight Spectral Attention Networks

TL;DR

This paper introduces an unsupervised deep learning framework for spectral demosaicing that reduces model complexity and improves performance on real-world hyperspectral images, validated on a new dataset.

Contribution

It proposes a novel unsupervised spectral demosaicing method with a lightweight spectral attention network and introduces the Mosaic25 dataset for benchmarking.

Findings

Outperforms conventional unsupervised methods in spectral fidelity.

Reduces model complexity and computational cost.

Effective on both synthetic and real-world datasets.

Abstract

This paper presents a deep learning-based spectral demosaicing technique trained in an unsupervised manner. Many existing deep learning-based techniques relying on supervised learning with synthetic images, often underperform on real-world images especially when the number of spectral bands increases. According to the characteristics of the spectral mosaic image, this paper proposes a mosaic loss function, the corresponding model structure, a transformation strategy, and an early stopping strategy, which form a complete unsupervised spectral demosaicing framework. A challenge in real-world spectral demosaicing is inconsistency between the model parameters and the computational resources of the imager. We reduce the complexity and parameters of the spectral attention module by dividing the spectral attention tensor into spectral attention matrices in the spatial dimension and spectral attention vector in the channel dimension, which is more suitable for unsupervised framework. This paper also presents Mosaic25, a real 25-band hyperspectral mosaic image dataset of various objects, illuminations, and materials for benchmarking. Extensive experiments on synthetic and real-world datasets demonstrate that the proposed method outperforms conventional unsupervised methods in terms of spatial distortion suppression, spectral fidelity, robustness, and computational cost.