SatMamba: Development of Foundation Models for Remote Sensing Imagery Using State Space Models
Chuc Man Duc, Hiromichi Fukui

TL;DR
SatMamba introduces a novel foundation model for remote sensing imagery that combines masked autoencoders with State Space Models, achieving linear computational scaling and improved efficiency for processing complex Earth observation data.
Contribution
The paper presents SatMamba, a new pretraining framework that integrates State Space Models with masked autoencoders, addressing ViT scalability issues in remote sensing applications.
Findings
Achieves linear computational scaling with input length.
Demonstrates superior performance on various remote sensing tasks.
Enables efficient processing of multispectral and multitemporal data.
Abstract
Foundation models refer to deep learning models pretrained on large unlabeled datasets through self-supervised algorithms. In the Earth science and remote sensing communities, there is growing interest in transforming the use of Earth observation data, including satellite and aerial imagery, through foundation models. Various foundation models have been developed for remote sensing, such as those for multispectral, high-resolution, and hyperspectral images, and have demonstrated superior performance on various downstream tasks compared to traditional supervised models. These models are evolving rapidly, with capabilities to handle multispectral, multitemporal, and multisensor data. Most studies use masked autoencoders in combination with Vision Transformers (ViTs) as the backbone for pretraining. While the models showed promising performance, ViTs face challenges, such as quadratic…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsTarget Tracking and Data Fusion in Sensor Networks · Advanced Computational Techniques and Applications · Bayesian Modeling and Causal Inference
