Using Neural Networks for Data Cleaning in Weather Datasets

TL;DR

This paper demonstrates that neural networks can effectively clean and improve the quality of weather datasets by reducing label noise, leading to better storm location predictions in climate data analysis.

Contribution

It introduces a neural network approach for data cleaning in weather datasets, showing improved storm location accuracy despite noisy labels.

Findings

Neural networks can denoise weather dataset labels effectively.

The approach outperforms original labels based on human preferences.

Neural networks reduce dataset misalignment issues in climate science data.

Abstract

In climate science, we often want to compare across different datasets. Difficulties can arise in doing this due to inevitable mismatches that arise between observational and reanalysis data, or even between different reanalyses. This misalignment can raise problems for any work that seeks to make inferences about one dataset from another. We considered tropical cyclone location as an example task with one dataset providing atmospheric conditions (ERA5) and another providing storm tracks (IBTrACS). We found that while the examples often aligned well, there were a considerable proportion (around 25%) which were not well aligned. We trained a neural network to map from the wind field to the storm location; in this setting misalignment in the datasets appears as "label noise" (i.e. the labelled storm location does not correspond to the underlying wind field). We found that this neural network trained only on the often noisy labels from IBTrACS had a denoising effect, and performed better than the IBTrACS labels themselves, as measured by human preferences. Remarkably, this even held true for training points, on which we might have expected the network to overfit to the IBTrACS predictions.