SDHSI-Net: Learning Better Representations for Hyperspectral Images via Self-Distillation

TL;DR

This paper introduces SDHSI-Net, a self-distillation-based deep learning model that enhances hyperspectral image classification by improving feature representation and accuracy without external teachers.

Contribution

It applies self-distillation to hyperspectral imaging, enforcing consistency between intermediate and final outputs to improve classification performance.

Findings

Significant accuracy improvements on benchmark datasets

Enhanced intra-class compactness and inter-class separability

Robustness to limited labeled data

Abstract

Hyperspectral image (HSI) classification presents unique challenges due to its high spectral dimensionality and limited labeled data. Traditional deep learning models often suffer from overfitting and high computational costs. Self-distillation (SD), a variant of knowledge distillation where a network learns from its own predictions, has recently emerged as a promising strategy to enhance model performance without requiring external teacher networks. In this work, we explore the application of SD to HSI by treating earlier outputs as soft targets, thereby enforcing consistency between intermediate and final predictions. This process improves intra-class compactness and inter-class separability in the learned feature space. Our approach is validated on two benchmark HSI datasets and demonstrates significant improvements in classification accuracy and robustness, highlighting the effectiveness of SD for spectral-spatial learning. Codes are available at https://github.com/Prachet-Dev-Singh/SDHSI.