COP-GEN: Latent Diffusion Transformer for Copernicus Earth Observation Data

TL;DR

COP-GEN is a novel multimodal latent diffusion transformer that models the joint distribution of diverse Earth observation data, enabling flexible, uncertainty-aware cross-modal generation and translation without retraining.

Contribution

It introduces a stochastic generative model for EO data that captures cross-modal structure and uncertainty, outperforming deterministic methods and providing a new benchmark for evaluation.

Findings

COP-GEN covers 90% of the real observation manifold on the benchmark.

It maintains strong peak fidelity across modalities.

It enables zero-shot cross-modal translation without retraining.

Abstract

Earth observation applications increasingly rely on data from multiple sensors, including optical, radar, elevation, and land-cover. Relationships between modalities are fundamental for data integration but are inherently non-injective: identical conditioning information can correspond to multiple physically plausible observations, and should be parametrised as conditional distributions. Deterministic models, by contrast, collapse toward conditional means and fail to represent the uncertainty and variability required for tasks such as data completion and cross-sensor translation. We introduce COP-GEN, a multimodal latent diffusion transformer that models the joint distribution of heterogeneous EO modalities at their native spatial resolutions. By parameterising cross-modal mappings as conditional distributions, COP-GEN enables flexible any-to-any conditional generation, including zero-shot modality translation without task-specific retraining. Experiments show that COP-GEN generates diverse yet physically consistent realisations while maintaining strong peak fidelity across optical, radar, and elevation modalities. Qualitative and quantitative analyses demonstrate that the model captures meaningful cross-modal structure and adapts its output uncertainty as conditioning information increases. We release a stochastic benchmark built from multi-temporal Sentinel-2 observations that enables distribution-level comparison of generative EO models. On this benchmark, COP-GEN covers 90% of the real observation manifold and 63% of its per-band reflectance range, while the strongest competing method collapses to 2.8% and 18%, respectively. These results highlight the importance of stochastic generative modeling for EO and motivate evaluation protocols beyond single-reference, pointwise metrics. Website: https://miquel-espinosa.github.io/cop-gen