Better Together: Evaluating the Complementarity of Earth Embedding Models

TL;DR

This paper introduces a new way to evaluate Earth embedding models by measuring how their fusion improves performance over individual models, revealing their complementary strengths across various tasks.

Contribution

The paper proposes an embedding complementarity index and demonstrates that fused Earth embeddings often outperform single models, highlighting the importance of model combination.

Findings

Fused embeddings outperform the best single model in four out of six tasks.

Complementarity varies by task and location, influencing fusion effectiveness.

Spatial scale of land cover classes partially explains complementarity in land cover regression.

Abstract

Earth embedding models transform Earth observation data into embeddings uniquely tied to locations on the Earth's surface. These models are typically evaluated in isolation, comparing the downstream task performance across different Earth embeddings. However, spatially aligned embeddings can naturally be fused, providing richer information per location, a capability that isolated evaluations fail to capture. We therefore propose assessing Earth embeddings by their complementarity: the performance gain of fused embeddings over the best single-model baseline. To operationalise this, we introduce an embedding complementarity index applicable to any embedding and task, and evaluate four Earth embedding models (AlphaEarth, Tessera, GeoCLIP, SatCLIP) in isolation, in all pairs, and jointly across six downstream tasks. Fused embeddings outperform the best single model in four out of six tasks, confirming that single-embedding evaluations often underestimate Earth embedding capabilities. Complementarity proves both task- and location-dependent. Further, for a land cover regression task, we find that complementarity is partially determined by the spatial scale of land cover classes. Complementarity reframes Earth embeddings: the greatest future gains may come not from any single Earth embedding model, but from combinations that are better together.