Valid Error Bars for Neural Weather Models using Conformal Prediction

TL;DR

This paper introduces a conformal prediction framework that provides reliable uncertainty estimates for neural weather models without altering their structure, enhancing trust and interpretability of forecasts.

Contribution

It formalizes a model-agnostic, post-processing method for calibrated error bounds applicable across variables, lead times, and locations in neural weather forecasting.

Findings

Provides calibrated error bounds for all forecast variables

Applicable to both deterministic and probabilistic models

Computationally efficient with negligible additional cost

Abstract

Neural weather models have shown immense potential as inexpensive and accurate alternatives to physics-based models. However, most models trained to perform weather forecasting do not quantify the uncertainty associated with their forecasts. This limits the trust in the model and the usefulness of the forecasts. In this work we construct and formalise a conformal prediction framework as a post-processing method for estimating this uncertainty. The method is model-agnostic and gives calibrated error bounds for all variables, lead times and spatial locations. No modifications are required to the model and the computational cost is negligible compared to model training. We demonstrate the usefulness of the conformal prediction framework on a limited area neural weather model for the Nordic region. We further explore the advantages of the framework for deterministic and probabilistic models.