Boosting Spatial-Spectral Masked Auto-Encoder Through Mining Redundant Spectra for HSI-SAR/LiDAR Classification

TL;DR

This paper introduces a novel masking strategy called Mining Redundant Spectra (MRS) for masked auto-encoders in hyperspectral image classification, which selectively masks similar spectral bands to enhance model learning and accuracy.

Contribution

The paper proposes the MRS strategy that improves spectral band masking by focusing on spectral similarity, addressing redundancy issues in MIM-based HSI-SAR/LiDAR classification.

Findings

MRS improves classification accuracy on Berlin and Houston 2018 datasets.

Selective masking of similar spectral bands enhances representation learning.

Experimental results validate the effectiveness of the MRS strategy.

Abstract

Although recent masked image modeling (MIM)-based HSI-LiDAR/SAR classification methods have gradually recognized the importance of the spectral information, they have not adequately addressed the redundancy among different spectra, resulting in information leakage during the pretraining stage. This issue directly impairs the representation ability of the model. To tackle the problem, we propose a new strategy, named Mining Redundant Spectra (MRS). Unlike randomly masking spectral bands, MRS selectively masks them by similarity to increase the reconstruction difficulty. Specifically, a random spectral band is chosen during pretraining, and the selected and highly similar bands are masked. Experimental results demonstrate that employing the MRS strategy during the pretraining stage effectively improves the accuracy of existing MIM-based methods on the Berlin and Houston 2018 datasets.