Spatiotemporal Transformer for Imputing Sparse Data: A Deep Learning Approach

TL;DR

This paper presents a novel Spatiotemporal Transformer model designed to accurately impute missing values in sparse environmental datasets, especially soil moisture data, using attention mechanisms and self-supervised training.

Contribution

The paper introduces the ST-Transformer, a deep learning model that captures complex spatiotemporal correlations for improved data imputation in sparse datasets.

Findings

Outperforms existing imputation methods on soil moisture data

Effectively integrates spatiotemporal covariates for enhanced accuracy

Demonstrates broad applicability across different datasets

Abstract

Effective management of environmental resources and agricultural sustainability heavily depends on accurate soil moisture data. However, datasets like the SMAP/Sentinel-1 soil moisture product often contain missing values across their spatiotemporal grid, which poses a significant challenge. This paper introduces a novel Spatiotemporal Transformer model (ST-Transformer) specifically designed to address the issue of missing values in sparse spatiotemporal datasets, particularly focusing on soil moisture data. The ST-Transformer employs multiple spatiotemporal attention layers to capture the complex spatiotemporal correlations in the data and can integrate additional spatiotemporal covariates during the imputation process, thereby enhancing its accuracy. The model is trained using a self-supervised approach, enabling it to autonomously predict missing values from observed data points. Our model's efficacy is demonstrated through its application to the SMAP 1km soil moisture data over a 36 x 36 km grid in Texas. It showcases superior accuracy compared to well-known imputation methods. Additionally, our simulation studies on other datasets highlight the model's broader applicability in various spatiotemporal imputation tasks.