Predicting missing values in spatio-temporal satellite data

TL;DR

This paper introduces a new gap-fill algorithm for spatio-temporal satellite data that predicts missing values using neighborhood data, suitable for large datasets, and validated on MODIS NDVI data with up to 50% missing data.

Contribution

The paper presents a novel, scalable gap-fill method based on quantile regression and sorting, implemented in an open-source R package for flexible application.

Findings

Good performance in root mean squared prediction error

Effective handling of up to 50% missing data

Open-source implementation available for customization

Abstract

Remotely sensed data are sparse, which means that data have missing values, for instance due to cloud cover. This is problematic for applications and signal processing algorithms that require complete data sets. To address the sparse data issue, we present a new gap-fill algorithm. The proposed method predicts each missing value separately based on data points in a spatio-temporal neighborhood around the missing data point. The computational workload can be distributed among several computers, making the method suitable for large datasets. The prediction of the missing values and the estimation of the corresponding prediction uncertainties are based on sorting procedures and quantile regression. The algorithm was applied to MODIS NDVI data from Alaska and tested with realistic cloud cover scenarios featuring up to 50% missing data. Validation against established software showed that the proposed method has a good performance in terms of the root mean squared prediction error. The procedure is implemented and available in the open-source R package gapfill. We demonstrate the software performance with a real data example and show how it can be tailored to specific data. Due to the flexible software design, users can control and redesign major parts of the procedure with little effort. This makes it an interesting tool for gap-filling satellite data and for the future development of gap-fill procedures.