Multi-task deep learning for large-scale building detail extraction from high-resolution satellite imagery

TL;DR

This paper introduces MT-BR, a multi-task neural network for efficient, large-scale extraction of detailed building information from high-resolution satellite images, improving accuracy and consistency for urban analysis.

Contribution

The paper presents a novel multi-task neural network, MT-BR, with a strategic sampling scheme and augmentation techniques, enhancing large-scale building detail extraction from satellite imagery.

Findings

MT-BR outperforms existing methods in accuracy and efficiency.

The sampling scheme improves predictive performance without changing network architecture.

Application to Shanghai demonstrates practical utility in urban datasets.

Abstract

Understanding urban dynamics and promoting sustainable development requires comprehensive insights about buildings. While geospatial artificial intelligence has advanced the extraction of such details from Earth observational data, existing methods often suffer from computational inefficiencies and inconsistencies when compiling unified building-related datasets for practical applications. To bridge this gap, we introduce the Multi-task Building Refiner (MT-BR), an adaptable neural network tailored for simultaneous extraction of spatial and attributional building details from high-resolution satellite imagery, exemplified by building rooftops, urban functional types, and roof architectural types. Notably, MT-BR can be fine-tuned to incorporate additional building details, extending its applicability. For large-scale applications, we devise a novel spatial sampling scheme that strategically selects limited but representative image samples. This process optimizes both the spatial distribution of samples and the urban environmental characteristics they contain, thus enhancing extraction effectiveness while curtailing data preparation expenditures. We further enhance MT-BR's predictive performance and generalization capabilities through the integration of advanced augmentation techniques. Our quantitative results highlight the efficacy of the proposed methods. Specifically, networks trained with datasets curated via our sampling method demonstrate improved predictive accuracy relative to those using alternative sampling approaches, with no alterations to network architecture. Moreover, MT-BR consistently outperforms other state-of-the-art methods in extracting building details across various metrics. The real-world practicality is also demonstrated in an application across Shanghai, generating a unified dataset that encompasses both the spatial and attributional details of buildings.