Scaling Remote Sensing Foundation Models: Data Domain Tradeoffs at the Peta-Scale

TL;DR

This paper investigates the scaling behaviors of remote sensing foundation models trained on petascale high-resolution satellite data, revealing data limitations and providing insights for future model development and data strategies.

Contribution

It presents the first large-scale analysis of remote sensing foundation models at petascale, highlighting data limitations and practical training insights for high-resolution EO datasets.

Findings

Performance is data-limited even at petascale.

Scaling laws from natural images do not directly apply to remote sensing.

Insights inform data collection and compute strategies for future models.

Abstract

We explore the scaling behaviors of artificial intelligence to establish practical techniques for training foundation models on high-resolution electro-optical (EO) datasets that exceed the current state-of-the-art scale by orders of magnitude. Modern multimodal machine learning (ML) applications, such as generative artificial intelligence (GenAI) systems for image captioning, search, and reasoning, depend on robust, domain-specialized encoders for non-text modalities. In natural image domains where internet-scale data is plentiful, well-established scaling laws help optimize the joint scaling of model capacity, training compute, and dataset size. Unfortunately, these relationships are much less well understood in high-value domains like remote sensing (RS). Using over a quadrillion pixels of commercial satellite EO data and MITRE's Federal AI Sandbox, we train progressively larger vision transformer (ViT) backbones, report successes and failure modes observed at peta-scale, and analyze implications for bridging domain gaps across additional RS modalities. We observe that even at this scale, performance is consistent with a data-limited regime rather than a model parameter-limited one. These practical insights are intended to inform data collection strategies, compute budgets, and optimization schedules that advance the future development of frontier scale RS foundation models.