Fine-Grained Building Function Recognition from Street-View Images via Geometry-Aware Semi-Supervised Learning

TL;DR

This paper introduces a geometry-aware semi-supervised learning framework that leverages geometric relationships among multi-source data to improve fine-grained building function recognition from street-view images, achieving significant accuracy gains and cross-city applicability.

Contribution

It presents a novel semi-supervised approach that combines GIS and street-view data with geometric relationships for accurate, large-scale building function recognition across multiple cities.

Findings

Achieves 7.6% and 4.8% improvements over fully-supervised and semi-supervised methods.

Effective in cross-city scenarios with different building function systems.

Facilitates large-scale urban management with minimal annotation.

Abstract

In this work, we propose a geometry-aware semi-supervised framework for fine-grained building function recognition, utilizing geometric relationships among multi-source data to enhance pseudo-label accuracy in semi-supervised learning, broadening its applicability to various building function categorization systems. Firstly, we design an online semi-supervised pre-training stage, which facilitates the precise acquisition of building facade location information in street-view images. In the second stage, we propose a geometry-aware coarse annotation generation module. This module effectively combines GIS data and street-view data based on the geometric relationships, improving the accuracy of pseudo annotations. In the third stage, we combine the newly generated coarse annotations with the existing labeled dataset to achieve fine-grained functional recognition of buildings across multiple cities at a large scale. Extensive experiments demonstrate that our proposed framework exhibits superior performance in fine-grained functional recognition of buildings. Within the same categorization system, it achieves improvements of 7.6\% and 4.8\% compared to fully-supervised methods and state-of-the-art semi-supervised methods, respectively. Additionally, our method also performs well in cross-city scenarios, i.e., extending the model trained on OmniCity (New York) to new cities (i.e., Los Angeles and Boston) with different building function categorization systems. This study offers a new solution for large-scale multi-city applications with minimal annotation requirements, facilitating more efficient data updates and resource allocation in urban management.