Comparing and Contrasting DLWP Backbones on Navier-Stokes and Atmospheric Dynamics

TL;DR

This paper empirically compares various deep learning architectures for weather prediction, analyzing their performance on synthetic Navier-Stokes data and real-world weather data to identify the most suitable models for different forecast horizons.

Contribution

It provides a controlled, empirical comparison of DLWP models with different backbones on synthetic and real data, highlighting their strengths and limitations.

Findings

FNO performs well on synthetic Navier-Stokes data.

ConvLSTM and SwinTransformer excel in short-to-mid-range weather forecasts.

Graph-based models like GraphCast and Spherical FNO are superior for long-term, stable forecasts.

Abstract

A large number of Deep Learning Weather Prediction (DLWP) architectures -- based on various backbones, including U-Net, Transformer, Graph Neural Network, and Fourier Neural Operator (FNO) -- have demonstrated their potential at forecasting atmospheric states. However, due to differences in training protocols, forecast horizons, and data choices, it remains unclear which (if any) of these methods and architectures are most suitable for weather forecasting and for future model development. Here, we step back and provide a detailed empirical analysis, under controlled conditions, comparing and contrasting the most prominent DLWP models, along with their backbones. We accomplish this by predicting synthetic two-dimensional incompressible Navier-Stokes and real-world global weather dynamics. On synthetic data, we observe favorable performance of FNO, while on the real-world WeatherBench dataset, our results demonstrate the suitability of ConvLSTM and SwinTransformer for short-to-mid-ranged forecasts. For long-ranged weather rollouts of up to 50 years, we observe superior stability and physical soundness in architectures that formulate a spherical data representation, i.e., GraphCast and Spherical FNO. The code is available at https://github.com/amazon-science/dlwp-benchmark.