AI Models Still Lag Behind Traditional Numerical Models in Predicting Sudden-Turning Typhoons

TL;DR

This paper evaluates AI-based weather prediction models, specifically Pangu-Weather, and finds they outperform traditional models in general but struggle with rare, sudden-turning typhoon events, highlighting limitations in current AI approaches.

Contribution

The study reassesses Pangu-Weather's ability to predict extreme typhoon trajectories, revealing its limitations in forecasting rare sudden-turning events compared to NWP models.

Findings

Pangu-Weather outperforms NWP in general TC track prediction

AI models struggle with rare sudden-turning typhoons

Current AI models lag behind NWP in extreme event prediction

Abstract

Given the interpretability, accuracy, and stability of numerical weather prediction (NWP) models, current operational weather forecasting relies heavily on the NWP approach. In the past two years, the rapid development of Artificial Intelligence (AI) has provided an alternative solution for medium-range (1-10 days) weather forecasting. Bi et al. (2023) (hereafter Bi23) introduced the first AI-based weather prediction (AIWP) model in China, named Pangu-Weather, which offers fast prediction without compromising accuracy. In their work, Bi23 made notable claims regarding its effectiveness in extreme weather predictions. However, this claim lacks persuasiveness because the extreme nature of the two tropical cyclones (TCs) examples presented in Bi23, namely Typhoon Kong-rey and Typhoon Yutu, stems primarily from their intensities rather than their moving paths. Their claim may mislead into another meaning which is that Pangu-Weather works well in predicting unusual typhoon paths, which was not explicitly analyzed. Here, we reassess Pangu-Weather's ability to predict extreme TC trajectories from 2020-2024. Results reveal that while Pangu-Weather overall outperforms NWP models in predicting tropical cyclone (TC) tracks, it falls short in accurately predicting the rarely observed sudden-turning tracks, such as Typhoon Khanun in 2023. We argue that current AIWP models still lag behind traditional NWP models in predicting such rare extreme events in medium-range forecasts.