Evaluating the Predictability of Selected Weather Extremes with Aurora, an AI Weather Forecast Model

TL;DR

This paper evaluates Aurora, an AI weather model, demonstrating strong short-term predictability for weather extremes but revealing limitations in long-term surface-impact forecasts due to atmospheric dynamics.

Contribution

It provides a comprehensive event-based assessment of Aurora's skill across various weather extremes and identifies a predictability limit beyond 10 days.

Findings

Aurora has high short-term skill for tropical cyclones and temperature extremes.

Predictability declines significantly beyond 10 days, especially for surface impacts.

Large-scale patterns remain predictable at 14-21 days, but surface impacts regress to climatology.

Abstract

AI weather foundation models now achieve forecast skill comparable to numerical weather prediction at far lower computational cost, yet their predictability for high-impact extremes across dynamical regimes remains uncertain. We evaluate Aurora using an event-based framework spanning tropical cyclones, freezes, heatwaves, atmospheric rivers, and extreme precipitation at lead times from 1 to 21 days. Aurora demonstrates strong short-range (1-7 day) skill across event types, including competitive tropical cyclone track accuracy and high spatial agreement for temperature and moisture extremes. However, a consistent subseasonal failure mode emerges: while large-scale circulation patterns remain moderately skillful at 14-21 day leads, threshold-based extreme intensity collapses as fields regress toward climatology. This divergence indicates that Aurora retains synoptic-scale dynamical structure but loses surface-impact amplitude beyond 7-10 days. The practical predictability horizon for deterministic AI extreme-event forecasting therefore remains constrained by intrinsic atmospheric dynamics.