HeatPrompt: Zero-Shot Vision-Language Modeling of Urban Heat Demand from Satellite Images

TL;DR

HeatPrompt leverages zero-shot vision-language models to estimate urban heat demand from satellite images, enabling heat planning in data-scarce regions with high accuracy and minimal data requirements.

Contribution

This paper introduces HeatPrompt, a novel zero-shot vision-language framework that estimates heat demand from satellite images using pretrained models and semantic prompts, reducing reliance on detailed building data.

Findings

Achieved 93.7% R^2 uplift over baseline models.

Reduced mean absolute error by 30%.

Qualitative analysis aligns high-demand zones with high-impact tokens.

Abstract

Accurate heat-demand maps play a crucial role in decarbonizing space heating, yet most municipalities lack detailed building-level data needed to calculate them. We introduce HeatPrompt, a zero-shot vision-language energy modeling framework that estimates annual heat demand using semantic features extracted from satellite images, basic Geographic Information System (GIS), and building-level features. We feed pretrained Large Vision Language Models (VLMs) with a domain-specific prompt to act as an energy planner and extract the visual attributes such as roof age, building density, etc, from the RGB satellite image that correspond to the thermal load. A Multi-Layer Perceptron (MLP) regressor trained on these captions shows an $R^2$ uplift of 93.7% and shrinks the mean absolute error (MAE) by 30% compared to the baseline model. Qualitative analysis shows that high-impact tokens align with high-demand zones, offering lightweight support for heat planning in data-scarce regions.