ADASR: An Adversarial Auto-Augmentation Framework for Hyperspectral and Multispectral Data Fusion

TL;DR

This paper introduces ADASR, an adversarial auto-augmentation framework that automatically enhances training data diversity for hyperspectral and multispectral image fusion, improving model robustness and performance.

Contribution

The paper presents a novel adversarial augmentation method that jointly optimizes an augmentor and downsampling networks for hyperspectral and multispectral data fusion.

Findings

ADASR outperforms state-of-the-art methods on public datasets.

The framework effectively enriches data diversity for better fusion results.

Experimental results demonstrate improved robustness of the trained models.

Abstract

Deep learning-based hyperspectral image (HSI) super-resolution, which aims to generate high spatial resolution HSI (HR-HSI) by fusing hyperspectral image (HSI) and multispectral image (MSI) with deep neural networks (DNNs), has attracted lots of attention. However, neural networks require large amounts of training data, hindering their application in real-world scenarios. In this letter, we propose a novel adversarial automatic data augmentation framework ADASR that automatically optimizes and augments HSI-MSI sample pairs to enrich data diversity for HSI-MSI fusion. Our framework is sample-aware and optimizes an augmentor network and two downsampling networks jointly by adversarial learning so that we can learn more robust downsampling networks for training the upsampling network. Extensive experiments on two public classical hyperspectral datasets demonstrate the effectiveness of our ADASR compared to the state-of-the-art methods.