Towards Vision-Language Geo-Foundation Model: A Survey

TL;DR

This survey reviews the development of Vision-Language Geo-Foundation Models (VLGFMs), emphasizing their unique geospatial data integration, core technologies, applications, and future research directions in the field.

Contribution

It is the first comprehensive review of VLGFMs, systematically summarizing recent advances, core methodologies, and discussing future challenges in geospatial multimodal modeling.

Findings

VLGFMs leverage large-scale geospatial multimodal data.

Core technologies include specialized data construction and model architectures.

VLGFMs show promising applications in earth observation tasks.

Abstract

Vision-Language Foundation Models (VLFMs) have made remarkable progress on various multimodal tasks, such as image captioning, image-text retrieval, visual question answering, and visual grounding. However, most methods rely on training with general image datasets, and the lack of geospatial data leads to poor performance on earth observation. Numerous geospatial image-text pair datasets and VLFMs fine-tuned on them have been proposed recently. These new approaches aim to leverage large-scale, multimodal geospatial data to build versatile intelligent models with diverse geo-perceptive capabilities, which we refer to as Vision-Language Geo-Foundation Models (VLGFMs). This paper thoroughly reviews VLGFMs, summarizing and analyzing recent developments in the field. In particular, we introduce the background and motivation behind the rise of VLGFMs, highlighting their unique research significance. Then, we systematically summarize the core technologies employed in VLGFMs, including data construction, model architectures, and applications of various multimodal geospatial tasks. Finally, we conclude with insights, issues, and discussions regarding future research directions. To the best of our knowledge, this is the first comprehensive literature review of VLGFMs. We keep tracing related works at https://github.com/zytx121/Awesome-VLGFM.