Decision-oriented benchmarking to transform AI weather forecast access: Application to the Indian monsoon

TL;DR

This paper introduces a decision-oriented benchmarking framework for AI weather prediction models, demonstrated through Indian monsoon forecasting, to better serve decision-making needs of vulnerable populations.

Contribution

It develops a novel framework that integrates meteorology, AI, and social sciences to evaluate AI weather models based on decision-making relevance, exemplified by Indian monsoon predictions.

Findings

AI models predict monsoon onset weeks in advance with high accuracy.

The framework influenced a government initiative reaching 38 million farmers.

AI forecasts captured unusual monsoon pauses, aiding decision-making.

Abstract

Artificial intelligence weather prediction (AIWP) models now often outperform traditional physics-based models on common metrics while requiring orders-of-magnitude less computing resources and time. Open-access AIWP models thus hold promise as transformational tools for helping low- and middle-income populations make decisions in the face of high-impact weather shocks. Yet, current approaches to evaluating AIWP models focus mainly on aggregated meteorological metrics without considering local stakeholders' needs in decision-oriented, operational frameworks. Here, we introduce such a framework that connects meteorology, AI, and social sciences. As an example, we apply it to the 150-year-old problem of Indian monsoon forecasting, focusing on benefits to rain-fed agriculture, which is highly susceptible to climate change. AIWP models skillfully predict an agriculturally relevant onset index at regional scales weeks in advance when evaluated out-of-sample using deterministic and probabilistic metrics. This framework informed a government-led effort in 2025 to send 38 million Indian farmers AI-based monsoon onset forecasts, which captured an unusual weeks-long pause in monsoon progression. This decision-oriented benchmarking framework provides a key component of a blueprint for harnessing the power of AIWP models to help large vulnerable populations adapt to weather shocks in the face of climate variability and change.