Boosting Hyperspectral Image Classification with Gate-Shift-Fuse Mechanisms in a Novel CNN-Transformer Approach

TL;DR

This paper introduces a novel CNN-Transformer hybrid model with Gate-Shift-Fuse mechanisms for hyperspectral image classification, effectively combining local and global feature extraction to improve accuracy.

Contribution

It presents a new HSI classification framework integrating GSF and transformer blocks, enhancing feature extraction over existing CNN-only methods.

Findings

Achieves superior classification accuracy on four benchmark datasets.

Effectively combines CNN and transformer strengths for HSI analysis.

Demonstrates improved local and global feature extraction capabilities.

Abstract

During the process of classifying Hyperspectral Image (HSI), every pixel sample is categorized under a land-cover type. CNN-based techniques for HSI classification have notably advanced the field by their adept feature representation capabilities. However, acquiring deep features remains a challenge for these CNN-based methods. In contrast, transformer models are adept at extracting high-level semantic features, offering a complementary strength. This paper's main contribution is the introduction of an HSI classification model that includes two convolutional blocks, a Gate-Shift-Fuse (GSF) block and a transformer block. This model leverages the strengths of CNNs in local feature extraction and transformers in long-range context modelling. The GSF block is designed to strengthen the extraction of local and global spatial-spectral features. An effective attention mechanism module is also proposed to enhance the extraction of information from HSI cubes. The proposed method is evaluated on four well-known datasets (the Indian Pines, Pavia University, WHU-WHU-Hi-LongKou and WHU-Hi-HanChuan), demonstrating that the proposed framework achieves superior results compared to other models.