RAMEN: Resolution-Adjustable Multimodal Encoder for Earth Observation

TL;DR

RAMEN is a flexible, sensor-agnostic multimodal encoder for Earth observation data that allows explicit control over spatial resolution, enabling effective analysis across diverse EO modalities and resolutions.

Contribution

Introduces RAMEN, a resolution-adjustable multimodal encoder that learns a unified representation across heterogeneous EO data, with spatial resolution as a controllable parameter.

Findings

Outperforms state-of-the-art models on PANGAEA benchmark

Generalizes well to unseen sensor configurations

Enables explicit trade-offs between spatial detail and computational cost

Abstract

Earth observation (EO) data spans a wide range of spatial, spectral, and temporal resolutions, from high-resolution optical imagery to low resolution multispectral products or radar time series. While recent foundation models have improved multimodal integration for learning meaningful representations, they often expect fixed input resolutions or are based on sensor-specific encoders limiting generalization across heterogeneous EO modalities. To overcome these limitations we introduce RAMEN, a resolution-adjustable multimodal encoder that learns a shared visual representation across EO data in a fully sensor-agnostic manner. RAMEN treats the modality and spatial and temporal resolutions as key input data features, enabling coherent analysis across modalities within a unified latent space. Its main methodological contribution is to define spatial resolution as a controllable output parameter, giving users direct control over the desired level of detail at inference and allowing explicit trade-offs between spatial precision and computational cost. We train a single, unified transformer encoder reconstructing masked multimodal EO data drawn from diverse sources, ensuring generalization across sensors and resolutions. Once pretrained, RAMEN transfers effectively to both known and unseen sensor configurations and outperforms larger state-of-the-art models on the community-standard PANGAEA benchmark, containing various multi-sensor and multi-resolution downstream tasks. Our code and pretrained model are available at https://github.com/nicolashoudre/RAMEN.