SERT: A Transfomer Based Model for Spatio-Temporal Sensor Data with Missing Values for Environmental Monitoring

TL;DR

This paper introduces SERT, a transformer-based model designed for multivariate spatio-temporal forecasting with missing sensor data, outperforming existing methods without requiring data imputation.

Contribution

The paper presents SERT, a novel transformer model that handles missing values in multivariate spatio-temporal data for environmental monitoring, with competitive performance.

Findings

SERT outperforms state-of-the-art models in forecasting accuracy.

SERT naturally handles missing data without imputation.

The simpler SST-ANN provides interpretable results.

Abstract

Environmental monitoring is crucial to our understanding of climate change, biodiversity loss and pollution. The availability of large-scale spatio-temporal data from sources such as sensors and satellites allows us to develop sophisticated models for forecasting and understanding key drivers. However, the data collected from sensors often contain missing values due to faulty equipment or maintenance issues. The missing values rarely occur simultaneously leading to data that are multivariate misaligned sparse time series. We propose two models that are capable of performing multivariate spatio-temporal forecasting while handling missing data naturally without the need for imputation. The first model is a transformer-based model, which we name SERT (Spatio-temporal Encoder Representations from Transformers). The second is a simpler model named SST-ANN (Sparse Spatio-Temporal Artificial Neural Network) which is capable of providing interpretable results. We conduct extensive experiments on two different datasets for multivariate spatio-temporal forecasting and show that our models have competitive or superior performance to those at the state-of-the-art.