ToPT: Task-Oriented Prompt Tuning for Urban Region Representation Learning

TL;DR

ToPT introduces a novel two-stage framework for urban region representation learning that incorporates spatial priors and task-specific prompts, achieving state-of-the-art results across multiple urban computing tasks and cities.

Contribution

The paper presents ToPT, a new method combining spatial-aware embedding learning and task-oriented prompting to improve urban region representations.

Findings

Achieves up to 64.2% performance improvement on various tasks.

Effectively models inter-region spatial relationships with a Graphormer-based module.

Aligns region embeddings with task semantics using large language model prompts.

Abstract

Learning effective region embeddings from heterogeneous urban data underpins key urban computing tasks (e.g., crime prediction, resource allocation). However, prevailing two-stage methods yield task-agnostic representations, decoupling them from downstream objectives. Recent prompt-based approaches attempt to fix this but introduce two challenges: they often lack explicit spatial priors, causing spatially incoherent inter-region modeling, and they lack robust mechanisms for explicit task-semantic alignment. We propose ToPT, a two-stage framework that delivers spatially consistent fusion and explicit task alignment. ToPT consists of two modules: spatial-aware region embedding learning (SREL) and task-aware prompting for region embeddings (Prompt4RE). SREL employs a Graphormer-based fusion module that injects spatial priors-distance and regional centrality-as learnable attention biases to capture coherent, interpretable inter-region interactions. Prompt4RE performs task-oriented prompting: a frozen multimodal large language model (MLLM) processes task-specific templates to obtain semantic vectors, which are aligned with region embeddings via multi-head cross-attention for stable task conditioning. Experiments across multiple tasks and cities show state-of-the-art performance, with improvements of up to 64.2\%, validating the necessity and complementarity of spatial priors and prompt-region alignment. The code is available at https://github.com/townSeven/Prompt4RE.git.