LGEST: Dynamic Spatial-Spectral Expert Routing for Hyperspectral Image Classification

TL;DR

LGEST introduces a novel deep learning framework for hyperspectral image classification that effectively integrates local and global spectral-spatial features, addressing scale disparities and high-dimensional challenges to outperform existing methods.

Contribution

The paper proposes LGEST, combining a hierarchical autoencoder, a cross-attention expert feature pyramid, and a dynamic expert routing system for improved hyperspectral image classification.

Findings

LGEST outperforms state-of-the-art methods on four benchmark datasets.

The hierarchical autoencoder preserves neighborhood coherence in high-dimensional spaces.

Dynamic expert routing enhances feature discrimination and classification accuracy.

Abstract

Deep learning methods, including Convolutional Neural Networks, Transformers and Mamba, have achieved remarkable success in hyperspectral image (HSI) classification. Nevertheless, existing methods exhibit inflexible integration of local-global representations, inadequate handling of spectral-spatial scale disparities across heterogeneous bands, and susceptibility to the Hughes phenomenon under high-dimensional sample heterogeneity. To address these challenges, we propose Local-Global Expert Spatial-Spectral Transformer (LGEST), a novel framework that synergistically combines three key innovations. The LGEST first employs a Deep Spatial-Spectral Autoencoder (DSAE) to generate compact yet discriminative embeddings through hierarchical nonlinear compression, preserving 3D neighborhood coherence while mitigating information loss in high-dimensional spaces. Secondly, a Cross-Interactive Mixed Expert Feature Pyramid (CIEM-FPN) leverages cross-attention mechanisms and residual mixture-of-experts layers to dynamically fuse multi-scale features, adaptively weighting spectral discriminability and spatial saliency through learnable gating functions. Finally, a Local-Global Expert System (LGES) processes decomposed features via sparsely activated expert pairs: convolutional sub-experts capture fine-grained textures, while transformer sub-experts model long-range contextual dependencies, with a routing controller dynamically selecting experts based on real-time feature saliency. Extensive experiments on four benchmark datasets demonstrate that LGEST consistently outperforms state-of-the-art methods.