Attention based Dual-Branch Complex Feature Fusion Network for Hyperspectral Image Classification

TL;DR

This paper introduces a dual-branch neural network that combines real-valued and complex-valued processing of hyperspectral images, leveraging Fourier transforms and SE mechanisms to improve classification accuracy.

Contribution

The novel dual-branch architecture integrates RVNN and CVNN with Fourier transforms and SE blocks, enhancing hyperspectral image classification performance.

Findings

Outperforms state-of-the-art methods in accuracy metrics

Fourier transform stream adds valuable frequency information

SE mechanism improves overall accuracy by nearly 1%

Abstract

This research work presents a novel dual-branch model for hyperspectral image classification that combines two streams: one for processing standard hyperspectral patches using Real-Valued Neural Network (RVNN) and the other for processing their corresponding Fourier transforms using Complex-Valued Neural Network (CVNN). The proposed model is evaluated on the Pavia University and Salinas datasets. Results show that the proposed model outperforms state-of-the-art methods in terms of overall accuracy, average accuracy, and Kappa. Through the incorporation of Fourier transforms in the second stream, the model is able to extract frequency information, which complements the spatial information extracted by the first stream. The combination of these two streams improves the overall performance of the model. Furthermore, to enhance the model performance, the Squeeze and Excitation (SE) mechanism has been utilized. Experimental evidence show that SE block improves the models overall accuracy by almost 1\%.