Efficient data-driven gap filling of satellite image time series using deep neural networks with partial convolutions

TL;DR

This paper introduces a deep learning method using 3D partial convolutions to efficiently fill gaps in satellite image time series, enabling faster processing of large datasets with comparable accuracy to statistical methods.

Contribution

It demonstrates how spatiotemporal partial convolutions can be integrated into neural networks for satellite data gap filling, improving speed and ease of use.

Findings

Prediction errors comparable to statistical approaches

Prediction computation times up to 1000 times faster

Open-source implementation available

Abstract

The abundance of gaps in satellite image time series often complicates the application of deep learning models such as convolutional neural networks for spatiotemporal modeling. Based on previous work in computer vision on image inpainting, this paper shows how three-dimensional spatiotemporal partial convolutions can be used as layers in neural networks to fill gaps in satellite image time series. To evaluate the approach, we apply a U-Net-like model on incomplete image time series of quasi-global carbon monoxide observations from the Sentinel-5P satellite. Prediction errors were comparable to two considered statistical approaches while computation times for predictions were up to three orders of magnitude faster, making the approach applicable to process large amounts of satellite data. Partial convolutions can be added as layers to other types of neural networks, making it relatively easy to integrate with existing deep learning models. However, the approach does not quantify prediction errors and further research is needed to understand and improve model transferability. The implementation of spatiotemporal partial convolutions and the U-Net-like model is available as open-source software.