Deep Learning-based Scalable Image-to-3D Facade Parser for Generating Thermal 3D Building Models

TL;DR

This paper introduces SI3FP, a deep learning-based pipeline that efficiently generates detailed thermal 3D building models from images, aiding early-stage renovation planning with high accuracy and scalability.

Contribution

The paper presents a novel scalable image-to-3D facade parser that directly models geometric primitives, improving accuracy and supporting diverse data sources for thermal building modeling.

Findings

Achieved approximately 5% error in window-to-wall ratio estimates.

Supports both sparse and dense image data sources.

Facilitates large-scale energy renovation planning.

Abstract

Renovating existing buildings is essential for climate impact. Early-phase renovation planning requires simulations based on thermal 3D models at Level of Detail (LoD) 3, which include features like windows. However, scalable and accurate identification of such features remains a challenge. This paper presents the Scalable Image-to-3D Facade Parser (SI3FP), a pipeline that generates LoD3 thermal models by extracting geometries from images using both computer vision and deep learning. Unlike existing methods relying on segmentation and projection, SI3FP directly models geometric primitives in the orthographic image plane, providing a unified interface while reducing perspective distortions. SI3FP supports both sparse (e.g., Google Street View) and dense (e.g., hand-held camera) data sources. Tested on typical Swedish residential buildings, SI3FP achieved approximately 5% error in window-to-wall ratio estimates, demonstrating sufficient accuracy for early-stage renovation analysis. The pipeline facilitates large-scale energy renovation planning and has broader applications in urban development and planning.