EWMoE: An effective model for global weather forecasting with mixture-of-experts

TL;DR

EWMoE is a novel deep learning model for global weather forecasting that achieves high accuracy with less training data and computational resources by using a mixture-of-experts architecture and specialized components.

Contribution

The paper introduces EWMoE, a new weather forecasting model that combines 3D position embedding, MoE layers, and tailored loss functions to improve accuracy and efficiency over existing models.

Findings

EWMoE outperforms FourCastNet and ClimaX across all forecast times.

EWMoE achieves competitive results with Pangu-Weather and GraphCast.

Ablation studies confirm the effectiveness of the MoE architecture.

Abstract

Weather forecasting is a crucial task for meteorologic research, with direct social and economic impacts. Recently, data-driven weather forecasting models based on deep learning have shown great potential, achieving superior performance compared with traditional numerical weather prediction methods. However, these models often require massive training data and computational resources. In this paper, we propose EWMoE, an effective model for accurate global weather forecasting, which requires significantly less training data and computational resources. Our model incorporates three key components to enhance prediction accuracy: 3D absolute position embedding, a core Mixture-of-Experts (MoE) layer, and two specific loss functions. We conduct our evaluation on the ERA5 dataset using only two years of training data. Extensive experiments demonstrate that EWMoE outperforms current models such as FourCastNet and ClimaX at all forecast time, achieving competitive performance compared with the state-of-the-art models Pangu-Weather and GraphCast in evaluation metrics such as Anomaly Correlation Coefficient (ACC) and Root Mean Square Error (RMSE). Additionally, ablation studies indicate that applying the MoE architecture to weather forecasting offers significant advantages in improving accuracy and resource efficiency. Code is available at https://github.com/Tomoyi/EWMoE.