Using data assimilation tools to dissect GraphDOP

TL;DR

This paper adapts Data Assimilation diagnostic tools to Machine Learning models, specifically analyzing GraphDOP's interpretability and explainability in weather forecasting by revealing its physical relevance and observation impacts.

Contribution

It demonstrates how DA diagnostics can be transferred to ML models like GraphDOP, enhancing interpretability and explainability in weather prediction.

Findings

GraphDOP learns meteorological features and spatial relationships.

The model captures physically meaningful processes like storm movements.

Fusing conventional and satellite data improves Earth system representation.

Abstract

The Data Assimilation (DA) community has been developing various diagnostics to understand the importance of the observing system in accurately forecasting the weather. They usually rely on the ability to compute the derivatives of the physical model output with respect to its initial condition. For example, the Forecast Sensitivity-based Observation Impact (FSOI) estimates the impact on the forecast error of each observation processed in the DA system. This paper presents how these DA diagnostic tools are transferred to Machine Learning (ML) models, as their derivatives are readily available through automatic differentiation. We specifically explore the interpretability and explainability of the observation-driven GraphDOP model developed at the European Centre for Medium-Range Weather Forecasts (ECMWF). The interpretability study demonstrates the effectiveness of GraphDOP's sliding attention window to learn the meteorological features present in the observation datasets and to learn the spatial relationships between different regions. Making these relationships more transparent confirms that GraphDOP captures real, physically meaningful processes, such as the movement of storm systems. The explainability of GraphDOP is explored by applying the FSOI tool to study the impact of the different observations on the forecast error. This inspection reveals that GraphDOP creates an internal representation of the Earth system by combining the information from conventional and satellite observations.