Towards Urban General Intelligence: A Review and Outlook of Urban Foundation Models

TL;DR

This paper reviews the emerging field of Urban Foundation Models, defining their scope, classifying existing research, proposing a framework for development, and discussing their potential to transform urban systems and smart city applications.

Contribution

It introduces the concept of UFMs, provides a taxonomy based on urban data modalities, and proposes a framework to advance versatile and effective UFMs.

Findings

Classified existing research using a data-centric taxonomy.

Summarized current benchmarks and datasets for UFMs.

Discussed applications and transformative potential of UFMs in urban systems.

Abstract

The integration of machine learning techniques has become a cornerstone in the development of intelligent urban services, significantly contributing to the enhancement of urban efficiency, sustainability, and overall livability. Recent advancements in foundational models, such as ChatGPT, have introduced a paradigm shift within the fields of machine learning and artificial intelligence. These models, with their exceptional capacity for contextual comprehension, problem-solving, and task adaptability, present a transformative opportunity to reshape the future of smart cities and drive progress toward Urban General Intelligence (UGI). Despite increasing attention to Urban Foundation Models (UFMs), this rapidly evolving field faces critical challenges, including the lack of clear definitions, systematic reviews, and universalizable solutions. To address these issues, this paper first introduces the definition and concept of UFMs and highlights the distinctive challenges involved in their development. Furthermore, we present a data-centric taxonomy that classifies existing research on UFMs according to the various urban data modalities and types. In addition, we propose a prospective framework designed to facilitate the realization of versatile UFMs, aimed at overcoming the identified challenges and driving further progress in this field. Finally, this paper systematically summarizes and discusses existing benchmarks and datasets related to UFMs, and explores the wide-ranging applications of UFMs within urban contexts, illustrating their potential to significantly impact and transform urban systems. A comprehensive collection of relevant research papers and open-source resources have been collated and are continuously updated at: https://github.com/usail-hkust/Awesome-Urban-Foundation-Models.