Community Research Earth Digital Intelligence Twin (CREDIT)

TL;DR

CREDIT is a scalable, user-friendly AI framework for atmospheric modeling that enhances weather prediction accuracy and efficiency by supporting advanced models like WXFormer and FUXI, outperforming traditional systems.

Contribution

The paper introduces CREDIT, a flexible platform for training and deploying AI-based atmospheric models, and demonstrates its effectiveness with novel models like WXFormer and FUXI.

Findings

WXFormer and FUXI outperform IFS HRES in 10-day forecasts.

CREDIT enables scalable, modular AI model development for weather prediction.

Models trained on ERA5 data show improved accuracy and efficiency.

Abstract

Recent advancements in artificial intelligence (AI) for numerical weather prediction (NWP) have significantly transformed atmospheric modeling. AI NWP models outperform traditional physics-based systems, such as the Integrated Forecast System (IFS), across several global metrics while requiring fewer computational resources. However, existing AI NWP models face limitations related to training datasets and timestep choices, often resulting in artifacts that reduce model performance. To address these challenges, we introduce the Community Research Earth Digital Intelligence Twin (CREDIT) framework, developed at NSF NCAR. CREDIT provides a flexible, scalable, and user-friendly platform for training and deploying AI-based atmospheric models on high-performance computing systems. It offers an end-to-end pipeline for data preprocessing, model training, and evaluation, democratizing access to advanced AI NWP capabilities. We demonstrate CREDIT's potential through WXFormer, a novel deterministic vision transformer designed to predict atmospheric states autoregressively, addressing common AI NWP issues like compounding error growth with techniques such as spectral normalization, padding, and multi-step training. Additionally, to illustrate CREDIT's flexibility and state-of-the-art model comparisons, we train the FUXI architecture within this framework. Our findings show that both FUXI and WXFormer, trained on six-hourly ERA5 hybrid sigma-pressure levels, generally outperform IFS HRES in 10-day forecasts, offering potential improvements in efficiency and forecast accuracy. CREDIT's modular design enables researchers to explore various models, datasets, and training configurations, fostering innovation within the scientific community.