Spectral-Spatial Self-Supervised Learning for Few-Shot Hyperspectral Image Classification

TL;DR

This paper introduces S4L-FSC, a novel spectral-spatial self-supervised learning method that leverages heterogeneous and homogeneous datasets to improve few-shot hyperspectral image classification by learning spatial and spectral features.

Contribution

The paper proposes a new pretraining framework combining rotation-mirror SSL and masked spectral reconstruction SSL to enhance few-shot hyperspectral image classification.

Findings

Outperforms existing methods on four HSI datasets.

Effectively learns spatial geometric diversity and spectral dependencies.

Significantly improves classification accuracy in few-shot scenarios.

Abstract

Few-shot classification of hyperspectral images (HSI) faces the challenge of scarce labeled samples. Self-Supervised learning (SSL) and Few-Shot Learning (FSL) offer promising avenues to address this issue. However, existing methods often struggle to adapt to the spatial geometric diversity of HSIs and lack sufficient spectral prior knowledge. To tackle these challenges, we propose a method, Spectral-Spatial Self-Supervised Learning for Few-Shot Hyperspectral Image Classification (S4L-FSC), aimed at improving the performance of few-shot HSI classification. Specifically, we first leverage heterogeneous datasets to pretrain a spatial feature extractor using a designed Rotation-Mirror Self-Supervised Learning (RM-SSL) method, combined with FSL. This approach enables the model to learn the spatial geometric diversity of HSIs using rotation and mirroring labels as supervisory signals, while acquiring transferable spatial meta-knowledge through few-shot learning. Subsequently, homogeneous datasets are utilized to pretrain a spectral feature extractor via a combination of FSL and Masked Reconstruction Self-Supervised Learning (MR-SSL). The model learns to reconstruct original spectral information from randomly masked spectral vectors, inferring spectral dependencies. In parallel, FSL guides the model to extract pixel-level discriminative features, thereby embedding rich spectral priors into the model. This spectral-spatial pretraining method, along with the integration of knowledge from heterogeneous and homogeneous sources, significantly enhances model performance. Extensive experiments on four HSI datasets demonstrate the effectiveness and superiority of the proposed S4L-FSC approach for few-shot HSI classification.