Nearest Neighbor-Based Contrastive Learning for Hyperspectral and LiDAR Data Classification

TL;DR

This paper introduces NNCNet, a novel self-supervised contrastive learning framework that leverages nearest neighbor data augmentation and bilinear attention to improve hyperspectral and LiDAR data classification accuracy.

Contribution

The paper proposes a new contrastive learning network with neighbor-based augmentation and high-order feature interaction for multisource data classification.

Findings

Outperforms state-of-the-art methods on four datasets

Effectively exploits unlabeled data for better representations

Enhances semantic alignment between heterogeneous data sources

Abstract

The joint hyperspectral image (HSI) and LiDAR data classification aims to interpret ground objects at more detailed and precise level. Although deep learning methods have shown remarkable success in the multisource data classification task, self-supervised learning has rarely been explored. It is commonly nontrivial to build a robust self-supervised learning model for multisource data classification, due to the fact that the semantic similarities of neighborhood regions are not exploited in existing contrastive learning framework. Furthermore, the heterogeneous gap induced by the inconsistent distribution of multisource data impedes the classification performance. To overcome these disadvantages, we propose a Nearest Neighbor-based Contrastive Learning Network (NNCNet), which takes full advantage of large amounts of unlabeled data to learn discriminative feature representations. Specifically, we propose a nearest neighbor-based data augmentation scheme to use enhanced semantic relationships among nearby regions. The intermodal semantic alignments can be captured more accurately. In addition, we design a bilinear attention module to exploit the second-order and even high-order feature interactions between the HSI and LiDAR data. Extensive experiments on four public datasets demonstrate the superiority of our NNCNet over state-of-the-art methods. The source codes are available at \url{https://github.com/summitgao/NNCNet}.