Calibrated Conformal Prediction Intervals for Microphysical Process Rates

TL;DR

This paper demonstrates that conformal prediction methods can produce well-calibrated, sharp uncertainty intervals for machine learning emulators of atmospheric microphysical process rates, aiding climate and weather modeling.

Contribution

It applies and compares split conformal prediction and conformalized quantile regression to microphysical process rate emulators, highlighting the advantages of the latter for climate variable uncertainty quantification.

Findings

Both methods produce well-calibrated prediction intervals.

Conformalized quantile regression offers more consistent intervals across different scales.

The methods are effective for uncertainty quantification in climate-related emulators.

Abstract

Conformal prediction can yield statistically valid prediction intervals for any regression model, with no model modifications and small computational costs. To assess its practical value, we apply conformal methods to quantify uncertainty in machine learning emulators of six microphysical process rates. Microphysical process rates describe small-scale processes in atmospheric clouds such as precipitation formation and aerosol-cloud interactions, and help understand weather and climate. The emulators are trained on simulation output from the ICOsahedral Nonhydrostatic (ICON) model in a limited-area numerical weather prediction configuration. We compare split conformal prediction for deterministic emulators with conformalized quantile regression for quantile regression emulators. Both conformal prediction methods yield well-calibrated and sharp prediction intervals on average, but conformalized quantile regression provides more consistent intervals across several orders of magnitude, making it preferable for the uncertainty quantification of climate variables.