Sonny: Breaking the Compute Wall in Medium-Range Weather Forecasting

TL;DR

Sonny is an efficient hierarchical transformer model for medium-range weather forecasting that achieves competitive accuracy with significantly reduced compute requirements, making advanced weather prediction more accessible.

Contribution

Introduces Sonny, a hierarchical transformer with a two-stage design that balances performance and computational efficiency for medium-range weather forecasting.

Findings

Achieves competitive forecast skill on WeatherBench2.

Operates within reasonable compute budgets, training on a single GPU in 5.5 days.

Outperforms FastNet, especially at extended tropical lead times.

Abstract

Weather forecasting is a fundamental problem for protecting lives and infrastructure from high-impact atmospheric events. Recently, data-driven weather forecasting methods based on deep learning have demonstrated strong performance, often reaching accuracy levels competitive with operational numerical systems. However, many existing models rely on large-scale training regimes and compute-intensive architectures, which raises the practical barrier for academic groups with limited compute resources. Here we introduce Sonny, an efficient hierarchical transformer that achieves competitive medium-range forecasting performance while remaining feasible within reasonable compute budgets. At the core of Sonny is a two-stage StepsNet design: a narrow slow path first models large-scale atmospheric dynamics, and a subsequent full-width fast path integrates thermodynamic interactions. To stabilize medium-range rollout without an additional fine-tuning stage, we apply exponential moving average (EMA) during training. On WeatherBench2, Sonny yields robust medium-range forecast skill, remains competitive with operational baselines, and demonstrates clear advantages over FastNet, particularly at extended tropical lead times. In practice, Sonny can be trained to convergence on a single NVIDIA A40 GPU in approximately 5.5 days.