Improving Region Representation Learning from Urban Imagery with Noisy Long-Caption Supervision

TL;DR

UrbanLN is a novel pre-training framework that enhances urban region representation learning by aligning long captions with visual features and effectively filtering noise using multi-model collaboration and self-distillation.

Contribution

The paper introduces UrbanLN, a new method that improves alignment of long textual descriptions with visual data and mitigates noise in caption generation for urban imagery.

Findings

UrbanLN outperforms existing methods on multiple downstream tasks.

The proposed noise suppression strategies improve robustness in noisy caption scenarios.

Extensive experiments validate the effectiveness across four real-world cities.

Abstract

Region representation learning plays a pivotal role in urban computing by extracting meaningful features from unlabeled urban data. Analogous to how perceived facial age reflects an individual's health, the visual appearance of a city serves as its "portrait", encapsulating latent socio-economic and environmental characteristics. Recent studies have explored leveraging Large Language Models (LLMs) to incorporate textual knowledge into imagery-based urban region representation learning. However, two major challenges remain: i) difficulty in aligning fine-grained visual features with long captions, and ii) suboptimal knowledge incorporation due to noise in LLM-generated captions. To address these issues, we propose a novel pre-training framework called UrbanLN that improves Urban region representation learning through Long-text awareness and Noise suppression. Specifically, we introduce an information-preserved stretching interpolation strategy that aligns long captions with fine-grained visual semantics in complex urban scenes. To effectively mine knowledge from LLM-generated captions and filter out noise, we propose a dual-level optimization strategy. At the data level, a multi-model collaboration pipeline automatically generates diverse and reliable captions without human intervention. At the model level, we employ a momentum-based self-distillation mechanism to generate stable pseudo-targets, facilitating robust cross-modal learning under noisy conditions. Extensive experiments across four real-world cities and various downstream tasks demonstrate the superior performance of our UrbanLN.