# Fusion of Heterogeneous Earth Observation Data for the Classification of Local Climate Zones

## TL;DR

This paper introduces a new data fusion framework combining satellite imagery and OSM data for local climate zone classification, outperforming traditional methods in accuracy and stability across diverse datasets.

## Contribution

It presents a novel model-level fusion approach that separately processes heterogeneous data sources and introduces a method for detecting OSM building incompleteness.

## Key findings

- Achieves over 6% higher accuracy than baseline.
- More stable performance across spectral diversities.
- Effective in large-scale, diverse geographic regions.

## Abstract

This paper proposes a novel framework for fusing multi-temporal, multispectral satellite images and OpenStreetMap (OSM) data for the classification of local climate zones (LCZs). Feature stacking is the most commonly-used method of data fusion but does not consider the heterogeneity of multimodal optical images and OSM data, which becomes its main drawback. The proposed framework processes two data sources separately and then combines them at the model level through two fusion models (the landuse fusion model and building fusion model), which aim to fuse optical images with landuse and buildings layers of OSM data, respectively. In addition, a new approach to detecting building incompleteness of OSM data is proposed. The proposed framework was trained and tested using data from the 2017 IEEE GRSS Data Fusion Contest, and further validated on one additional test set containing test samples which are manually labeled in Munich and New York. Experimental results have indicated that compared to the feature stacking-based baseline framework the proposed framework is effective in fusing optical images with OSM data for the classification of LCZs with high generalization capability on a large scale. The classification accuracy of the proposed framework outperforms the baseline framework by more than 6% and 2%, while testing on the test set of 2017 IEEE GRSS Data Fusion Contest and the additional test set, respectively. In addition, the proposed framework is less sensitive to spectral diversities of optical satellite images and thus achieves more stable classification performance than state-of-the art frameworks.