CMTNet: Convolutional Meets Transformer Network for Hyperspectral Images Classification

TL;DR

CMTNet combines CNN and Transformer architectures to improve hyperspectral image classification by capturing both local and global features, outperforming existing methods across multiple datasets.

Contribution

The paper introduces CMTNet, a novel hybrid model that integrates CNN and Transformer modules with a multi-output constraint for enhanced hyperspectral image classification.

Findings

CMTNet achieves superior accuracy compared to state-of-the-art models.

The dual-branch structure effectively captures diverse spectral-spatial features.

Extensive experiments validate the model's robustness across multiple datasets.

Abstract

Hyperspectral remote sensing (HIS) enables the detailed capture of spectral information from the Earth's surface, facilitating precise classification and identification of surface crops due to its superior spectral diagnostic capabilities. However, current convolutional neural networks (CNNs) focus on local features in hyperspectral data, leading to suboptimal performance when classifying intricate crop types and addressing imbalanced sample distributions. In contrast, the Transformer framework excels at extracting global features from hyperspectral imagery. To leverage the strengths of both approaches, this research introduces the Convolutional Meet Transformer Network (CMTNet). This innovative model includes a spectral-spatial feature extraction module for shallow feature capture, a dual-branch structure combining CNN and Transformer branches for local and global feature extraction, and a multi-output constraint module that enhances classification accuracy through multi-output loss calculations and cross constraints across local, international, and joint features. Extensive experiments conducted on three datasets (WHU-Hi-LongKou, WHU-Hi-HanChuan, and WHU-Hi-HongHu) demonstrate that CTDBNet significantly outperforms other state-of-the-art networks in classification performance, validating its effectiveness in hyperspectral crop classification.