Band Prompting Aided SAR and Multi-Spectral Data Fusion Framework for Local Climate Zone Classification

TL;DR

This paper introduces a novel band prompting aided data fusion framework for Local Climate Zone classification that leverages textual prompts and a multivariate supervised matrix to improve the integration of SAR and multi-spectral data.

Contribution

It proposes a new fusion framework with band group prompting and MSM-based training to enhance LCZ classification performance using SAR and multi-spectral data.

Findings

Demonstrates improved LCZ classification accuracy.

Shows effectiveness of band prompting in data fusion.

Outperforms existing fusion methods.

Abstract

Local climate zone (LCZ) classification is of great value for understanding the complex interactions between urban development and local climate. Recent studies have increasingly focused on the fusion of synthetic aperture radar (SAR) and multi-spectral data to improve LCZ classification performance. However, it remains challenging due to the distinct physical properties of these two types of data and the absence of effective fusion guidance. In this paper, a novel band prompting aided data fusion framework is proposed for LCZ classification, namely BP-LCZ, which utilizes textual prompts associated with band groups to guide the model in learning the physical attributes of different bands and semantics of various categories inherent in SAR and multi-spectral data to augment the fused feature, thus enhancing LCZ classification performance. Specifically, a band group prompting (BGP) strategy is introduced to align the visual representation effectively at the level of band groups, which also facilitates a more adequate extraction of semantic information of different bands with textual information. In addition, a multivariate supervised matrix (MSM) based training strategy is proposed to alleviate the problem of positive and negative sample confusion by completing the supervised information. The experimental results demonstrate the effectiveness and superiority of the proposed data fusion framework.