Error in ERA5 2m Temperature identified using GraphCast

TL;DR

This study identifies a recurrent error in ERA5 reanalysis data's 2m temperature over Ethiopia, caused by data assimilation issues, which affects machine learning weather prediction models trained on ERA5.

Contribution

The paper reveals a specific, systematic error in ERA5 reanalysis data affecting 2m temperature over Ethiopia, and demonstrates its impact on machine learning weather prediction models.

Findings

Error in ERA5 2m temperature over Ethiopia at 0600 UTC.

The error arises from the 2D optimal interpolation data assimilation process.

GraphCast's forecast skill is slightly degraded by these ERA5 errors.

Abstract

Reanalyses such as ERA5 have long been foundational for weather and climate science. They have also found a new use case, as training and verification data for machine-learnt weather prediction (MLWP) models. Here we compare short-lead time (6h) forecasts from the MLWP model GraphCast against ERA5. In doing so, we identify a recurrent, spatially coherent error in 2m Temperature centred on the Ethiopian Highlands, that occurs predominantly at 0600 UTC. We show that these error events are not an error in the forecast from GraphCast, but are in fact an error in ERA5, and are also present in the ECMWF operational analysis. They arise from the 2D optimal interpolation procedure, when surface reports are assimilated that are temporally displaced compared to the background forecast. This produces spuriously warm analysis increments over Ethiopia on approximately 7\% of dates at 0600 UTC across the reanalysis record. The spread from the ensemble of data assimilation partially flags these cases but is underdispersive. We assess the impact on GraphCast, which was trained on ERA5. While GraphCast can largely ignore these unphysical error events, a small systematic degradation in forecast skill over the region is observed. We discuss implications for using reanalysis as truth in machine learning training and verification, and recommend simple changes to reduce such artefacts in future analyses.