What if? Numerical weather prediction at the crossroads

TL;DR

This paper explores the transformative impact of machine learning and digital technology on operational weather prediction, emphasizing new opportunities, challenges, and strategic considerations for future developments.

Contribution

It provides a forward-looking analysis of how operational weather centers can adapt to technological advances and data-driven approaches, outlining potential scenarios and strategic directions.

Findings

Machine learning enables creation of emulators outperforming traditional models.

Operational datasets from analyses and reanalyses are crucial for training ML models.

Integration of computational science accelerates progress in weather prediction.

Abstract

This paper provides an outlook on the future of operational weather prediction given the recent evolution in science, computing and machine learning. In many parts, this evolution strongly deviates from the strategy operational centres have formulated only several years ago. New opportunities in digital technology have greatly accelerated progress, and the full integration of computational science in numerical weather prediction centres is common knowledge now. Within the last few years, a vast machine learning research community has emerged for creating new and tailor-made products, accelerating processing and - most of all - creating emulators for the entire production of global forecasts that outperform traditional systems at the spatial resolution of the training data. In this context, the role of both numerical models and observations is changing from being equation to data driven. Analyses and reanalyses are becoming the new currency for training machine learning, and operational centres are in a powerful position as they generate these datasets based on decades worth of experience. This environment creates incredible opportunities to progress much faster than in the past but also uncertainties about what the strategic implications on defining cost-effective and sustainable research and operations are, and how to achieve sufficient high-performance computing and data handling capacities. It will take individual national public services a while to understand what to focus on and how to coordinate their substantial investments in staff and infrastructure at institutional, national and international level. This paper addresses this new situation operational weather prediction finds itself in through formulating the most likely "what if?" scenarios for the near future and provides an outline for how weather centres could adapt.