A Unified Framework for Next-Gen Urban Forecasting via LLM-driven Dependency Retrieval and GeoTransformer

TL;DR

This paper introduces a comprehensive urban forecasting framework that integrates spatial data and natural language prompts, utilizing a GeoTransformer architecture with geospatial attention to improve prediction accuracy across various urban tasks.

Contribution

It presents a novel, modular framework combining dependency retrieval and GeoTransformer for improved urban forecasting, addressing limitations of existing methods.

Findings

Demonstrates strong generalization across six urban forecasting tasks.

Validates effectiveness through quantitative and qualitative analysis.

Supports diverse representation methods and minimal input scenarios.

Abstract

Urban forecasting has increasingly benefited from high-dimensional spatial data through two primary approaches: graph-based methods that rely on predefined spatial structures, and region-based methods that focus on learning expressive urban representations. Although these methods have laid a strong foundation, they either rely heavily on structured spatial data, struggle to adapt to task-specific dependencies, or fail to integrate holistic urban context. Moreover, no existing framework systematically integrates these two paradigms and overcomes their respective limitations. To address this gap, we propose a novel, unified framework for high-dimensional urban forecasting, composed of three key components: (1) the Urban Region Representation Module that organizes latent embeddings and semantic descriptions for each region, (2) the Task-aware Dependency Retrieval module that selects relevant context regions based on natural language prompts, and (3) the Prediction Module, exemplified by our proposed GeoTransformer architecture, which adopts a novel geospatial attention mechanism to incorporate spatial proximity and information entropy as priors. Our framework is modular, supports diverse representation methods and forecasting models, and can operate even with minimal input. Quantitative experiments and qualitative analysis across six urban forecasting tasks demonstrate strong task generalization and validate the framework's effectiveness.