HighFM: Towards a Foundation Model for Learning Representations from High-Frequency Earth Observation Data

TL;DR

HighFM introduces a foundation model trained on high-frequency geostationary satellite data, improving real-time disaster monitoring and detection through enhanced spatiotemporal representations.

Contribution

This work adapts the SatMAE framework to high temporal resolution EO data, enabling effective foundation models for rapid disaster response tasks.

Findings

Pretrained models outperform traditional baselines in cloud masking and fire detection.

Fine grained temporal encodings improve short-term variability capture.

Consistent accuracy and IoU gains demonstrate effectiveness of the approach.

Abstract

The increasing frequency and severity of climate related disasters have intensified the need for real time monitoring, early warning, and informed decision-making. Earth Observation (EO), powered by satellite data and Machine Learning (ML), offers powerful tools to meet these challenges. Foundation Models (FMs) have revolutionized EO ML by enabling general-purpose pretraining on large scale remote sensing datasets. However most existing models rely on high-resolution satellite imagery with low revisit rates limiting their suitability for fast-evolving phenomena and time critical emergency response. In this work, we present HighFM, a first cut approach towards a FM for high temporal resolution, multispectral EO data. Leveraging over 2 TB of SEVIRI imagery from the Meteosat Second Generation (MSG) platform, we adapt the SatMAE masked autoencoding framework to learn robust spatiotemporal representations. To support real time monitoring, we enhance the original architecture with fine grained temporal encodings to capture short term variability. The pretrained models are then finetuned on cloud masking and active fire detection tasks. We benchmark our SEVIRI pretrained Vision Transformers against traditional baselines and recent geospatial FMs, demonstrating consistent gains across both balanced accuracy and IoU metrics. Our results highlight the potential of temporally dense geostationary data for real-time EO, offering a scalable path toward foundation models for disaster detection and tracking.