HSIGene: A Foundation Model For Hyperspectral Image Generation

TL;DR

HSIGene is a new foundation model for hyperspectral image generation that uses latent diffusion and multi-condition control to produce diverse, high-quality HSIs, addressing data scarcity and fidelity issues in the field.

Contribution

The paper introduces HSIGene, a novel latent diffusion-based foundation model supporting multi-condition control for hyperspectral image synthesis, along with a new data augmentation and super-resolution framework.

Findings

HSIGene generates realistic and diverse HSIs for downstream tasks.

The model improves spectral fidelity and spatial diversity in synthesized images.

Experiments show enhanced performance in denoising and super-resolution tasks.

Abstract

Hyperspectral image (HSI) plays a vital role in various fields such as agriculture and environmental monitoring. However, due to the expensive acquisition cost, the number of hyperspectral images is limited, degenerating the performance of downstream tasks. Although some recent studies have attempted to employ diffusion models to synthesize HSIs, they still struggle with the scarcity of HSIs, affecting the reliability and diversity of the generated images. Some studies propose to incorporate multi-modal data to enhance spatial diversity, but the spectral fidelity cannot be ensured. In addition, existing HSI synthesis models are typically uncontrollable or only support single-condition control, limiting their ability to generate accurate and reliable HSIs. To alleviate these issues, we propose HSIGene, a novel HSI generation foundation model which is based on latent diffusion and supports multi-condition control, allowing for more precise and reliable HSI generation. To enhance the spatial diversity of the training data while preserving spectral fidelity, we propose a new data augmentation method based on spatial super-resolution, in which HSIs are upscaled first, and thus abundant training patches could be obtained by cropping the high-resolution HSIs. In addition, to improve the perceptual quality of the augmented data, we introduce a novel two-stage HSI super-resolution framework, which first applies RGB bands super-resolution and then utilizes our proposed Rectangular Guided Attention Network (RGAN) for guided HSI super-resolution. Experiments demonstrate that the proposed model is capable of generating a vast quantity of realistic HSIs for downstream tasks such as denoising and super-resolution. The code and models are available at https://github.com/LiPang/HSIGene.