Densification and forecasting of Sentinel-2 time series from multimodal SAR and Optical satellite data using deep generative models

TL;DR

This paper introduces a deep generative model that densifies and forecasts Sentinel-2 satellite time series by integrating optical and SAR data, explicitly modeling uncertainty for improved Earth observation applications.

Contribution

The work presents a novel probabilistic deep learning framework that jointly uses multimodal satellite data to densify and predict future satellite images, addressing gaps in existing methods.

Findings

Effective densification of sparse time series demonstrated.

Successful forecasting of future satellite images shown.

Uncertainty modeling improves reliability of generated images.

Abstract

Optical satellite image time series are extensively used in many Earth observation applications, including agriculture, climate monitoring, and land surface analysis. However, clouds and swath edges result in irregular sampling along the temporal dimension, limiting continuous monitoring. To address this issue, a growing body of work has focused on temporal densification and reconstruction of satellite image time series, with the objective of filling missing or cloud-contaminated observations within the temporal extent of the available data. While these approaches improve temporal continuity, they are inherently restricted to the reconstruction of the gaps within the observed time periods, and do not address the prediction of future observations. This work proposes a probabilistic deep learning framework for the densification and forecasting of Sentinel-2 time series by generating optical images at arbitrary past or future dates. The approach leverages multimodal satellite data by jointly exploiting Sentinel-2 optical and Sentinel-1 SAR observations. Unlike most existing works, we propose to focus on the uncertainty of the generated images. Experimental results demonstrate effective densification and forecasting, on sparse and temporally misaligned time series.