Benchmarking Physics-Informed Time-Series Models for Operational Global Station Weather Forecasting

TL;DR

This paper introduces WEATHER-5K, a large-scale weather dataset, and PhysicsFormer, a physics-informed time-series model, to improve global weather forecasting and bridge the gap with operational systems.

Contribution

The paper presents WEATHER-5K dataset and PhysicsFormer model, combining physical constraints with Transformer architecture for enhanced weather prediction accuracy.

Findings

PhysicsFormer outperforms other TSF models on weather variables.

The dataset enables better training and evaluation of forecasting models.

Benchmark results highlight the gap between academic models and operational systems.

Abstract

The development of Time-Series Forecasting (TSF) models is often constrained by the lack of comprehensive datasets, especially in Global Station Weather Forecasting (GSWF), where existing datasets are small, temporally short, and spatially sparse. To address this, we introduce WEATHER-5K, a large-scale observational weather dataset that better reflects real-world conditions, supporting improved model training and evaluation. While recent TSF methods perform well on benchmarks, they lag behind operational Numerical Weather Prediction systems in capturing complex weather dynamics and extreme events. We propose PhysicsFormer, a physics-informed forecasting model combining a dynamic core with a Transformer residual to predict future weather states. Physical consistency is enforced via pressure-wind alignment and energy-aware smoothness losses, ensuring plausible dynamics while capturing complex temporal patterns. We benchmark PhysicsFormer and other TSF models against operational systems across several weather variables, extreme event prediction, and model complexity, providing a comprehensive assessment of the gap between academic TSF models and operational forecasting. The dataset and benchmark implementation are available at: https://github.com/taohan10200/WEATHER-5K.