A3CLNN: Spatial, Spectral and Multiscale Attention ConvLSTM Neural Network for Multisource Remote Sensing Data Classification

TL;DR

This paper introduces a dual-channel A3CLNN neural network that effectively combines hyperspectral images and LiDAR data using attention mechanisms and a novel fusion strategy, significantly improving multisource remote sensing classification.

Contribution

It presents a novel dual-channel A3CLNN with spatial, spectral, and multiscale attention for multisource data fusion and a stepwise training strategy, advancing remote sensing classification methods.

Findings

Outperforms state-of-the-art methods in classification accuracy

Effectively learns spectral and spatial features from multisource data

Demonstrates robustness across multiple datasets

Abstract

The problem of effectively exploiting the information multiple data sources has become a relevant but challenging research topic in remote sensing. In this paper, we propose a new approach to exploit the complementarity of two data sources: hyperspectral images (HSIs) and light detection and ranging (LiDAR) data. Specifically, we develop a new dual-channel spatial, spectral and multiscale attention convolutional long short-term memory neural network (called dual-channel A3CLNN) for feature extraction and classification of multisource remote sensing data. Spatial, spectral and multiscale attention mechanisms are first designed for HSI and LiDAR data in order to learn spectral- and spatial-enhanced feature representations, and to represent multiscale information for different classes. In the designed fusion network, a novel composite attention learning mechanism (combined with a three-level fusion strategy) is used to fully integrate the features in these two data sources. Finally, inspired by the idea of transfer learning, a novel stepwise training strategy is designed to yield a final classification result. Our experimental results, conducted on several multisource remote sensing data sets, demonstrate that the newly proposed dual-channel A3CLNN exhibits better feature representation ability (leading to more competitive classification performance) than other state-of-the-art methods.