Automated Quality Assessment of (Citizen) Weather Stations

TL;DR

This paper introduces a method to dynamically assess and improve the quality of citizen weather station data by optimizing Gaussian process regression with a genetic algorithm, significantly enhancing prediction accuracy.

Contribution

It presents a novel approach combining Gaussian process regression and genetic algorithms to evaluate and improve sensor data quality in real-time.

Findings

12.5% reduction in mean absolute error

Effective quality assessment for heterogeneous sensor data

Improved data reliability for citizen weather stations

Abstract

Today we have access to a vast amount of weather, air quality, noise or radioactivity data collected by individual around the globe. This volunteered geographic information often contains data of uncertain and of heterogeneous quality, in particular when compared to official in-situ measurements. This limits their application, as rigorous, work-intensive data cleaning has to be performed, which reduces the amount of data and cannot be performed in real-time. In this paper, we propose dynamically learning the quality of individual sensors by optimizing a weighted Gaussian process regression using a genetic algorithm. We chose weather stations as our use case as these are the most common VGI measurements. The evaluation is done for the south-west of Germany in August 2016 with temperature data from the Wunderground network and the Deutsche Wetter Dienst (DWD), in total 1561 stations. Using a 10-fold cross-validation scheme based on the DWD ground truth, we can show significant improvements of the predicted sensor reading. In our experiment we were obtain a 12.5% improvement on the mean absolute error.