CLAReSNet: When Convolution Meets Latent Attention for Hyperspectral Image Classification

TL;DR

CLAReSNet is a hybrid neural network architecture that combines convolutional feature extraction with transformer-style attention, significantly improving hyperspectral image classification accuracy and efficiency, especially under class imbalance.

Contribution

The paper introduces CLAReSNet, a novel hybrid model integrating multi-scale convolution, residual blocks, and spectral latent attention with adaptive token allocation for hyperspectral classification.

Findings

Achieves state-of-the-art accuracy on Indian Pines and Salinas datasets.

Reduces spectral attention complexity from quadratic to logarithmic scale.

Embeddings show superior class separation and intra-class compactness.

Abstract

Hyperspectral image (HSI) classification faces critical challenges, including high spectral dimensionality, complex spectral-spatial correlations, and limited training samples with severe class imbalance. While CNNs excel at local feature extraction and transformers capture long-range dependencies, their isolated application yields suboptimal results due to quadratic complexity and insufficient inductive biases. We propose CLAReSNet (Convolutional Latent Attention Residual Spectral Network), a hybrid architecture that integrates multi-scale convolutional extraction with transformer-style attention via an adaptive latent bottleneck. The model employs a multi-scale convolutional stem with deep residual blocks and an enhanced Convolutional Block Attention Module for hierarchical spatial features, followed by spectral encoder layers combining bidirectional RNNs (LSTM/GRU) with Multi-Scale Spectral Latent Attention (MSLA). MSLA reduces complexity from $\mathcal{O}(T^2D)$ to $\mathcal{O}(T\log(T)D)$ by adaptive latent token allocation (8-64 tokens) that scales logarithmically with the sequence length. Hierarchical cross-attention fusion dynamically aggregates multi-level representations for robust classification. Experiments conducted on the Indian Pines and Salinas datasets show state-of-the-art performance, achieving overall accuracies of 99.71% and 99.96%, significantly surpassing HybridSN, SSRN, and SpectralFormer. The learned embeddings exhibit superior inter-class separability and compact intra-class clustering, validating CLAReSNet's effectiveness under severe class imbalance.