Foundation Models in Remote Sensing: Evolving from Unimodality to Multimodality

TL;DR

This paper provides a comprehensive survey of foundation models in remote sensing, highlighting their evolution from unimodal to multimodal approaches and offering guidance for researchers to understand and apply these models effectively.

Contribution

It systematically reviews existing foundation models in RS, categorizes them into unimodal and multimodal, and offers practical guidance for training and applying these models.

Findings

Foundation models are crucial for advancing RS data analysis.

Transition from unimodal to multimodal models enhances capabilities.

Provides a tutorial for training and deploying RS foundation models.

Abstract

Remote sensing (RS) techniques are increasingly crucial for deepening our understanding of the planet. As the volume and diversity of RS data continue to grow exponentially, there is an urgent need for advanced data modeling and understanding capabilities to manage and interpret these vast datasets effectively. Foundation models present significant new growth opportunities and immense potential to revolutionize the RS field. In this paper, we conduct a comprehensive technical survey on foundation models in RS, offering a brand-new perspective by exploring their evolution from unimodality to multimodality. We hope this work serves as a valuable entry point for researchers interested in both foundation models and RS and helps them launch new projects or explore new research topics in this rapidly evolving area. This survey addresses the following three key questions: What are foundation models in RS? Why are foundation models needed in RS? How can we effectively guide junior researchers in gaining a comprehensive and practical understanding of foundation models in RS applications? More specifically, we begin by outlining the background and motivation, emphasizing the importance of foundation models in RS. We then review existing foundation models in RS, systematically categorizing them into unimodal and multimodal approaches. Additionally, we provide a tutorial-like section to guide researchers, especially beginners, on how to train foundation models in RS and apply them to real-world tasks. The survey aims to equip researchers in RS with a deeper and more efficient understanding of foundation models, enabling them to get started easily and effectively apply these models across various RS applications.