TerraFlow: Multimodal, Multitemporal Representation Learning for Earth Observation

TL;DR

TerraFlow is a new multimodal, multitemporal learning model for Earth observation that outperforms existing models in various temporal tasks and shows promise in disaster risk mapping.

Contribution

It introduces a sequence-aware training approach for multimodal, multitemporal Earth observation data, improving robustness and performance over state-of-the-art models.

Findings

Outperforms state-of-the-art models by up to 50% in F1 score.

Achieves 24% improvement in Brier score.

Demonstrates initial success in disaster risk map prediction.

Abstract

We propose TerraFlow, a novel approach to multimodal, multitemporal learning for Earth observation. TerraFlow builds on temporal training objectives that enable sequence-aware learning across space, time, and modality, while remaining robust to the variable-length inputs commonly encountered in real-world Earth observation data. Our experiments demonstrate superiority of TerraFlow over state-of-the-art foundation models for Earth observation across all temporal tasks of the GEO-Bench-2 benchmark. We additionally demonstrate that TerraFlow is able to make initial steps towards deep-learning based risk map prediction for natural disasters -- a task on which other state-of-the-art foundation models frequently collapse. TerraFlow outperforms state-of-the-art foundation models by up to 50% in F1 score and 24% in Brier score.