Pixels and Predictions: Potential of GPT-4V in Meteorological Imagery Analysis and Forecast Communication

TL;DR

This paper evaluates GPT-4V's ability to interpret meteorological imagery and communicate weather hazards, highlighting its potential and limitations in forecasting and multilingual hazard communication.

Contribution

It provides an empirical assessment of GPT-4V's effectiveness in meteorological analysis and multilingual hazard communication, emphasizing the need for cautious integration and human oversight.

Findings

GPT-4V's weather outlooks align well with human forecasts.

Spanish hazard summaries are poorly translated, losing critical idiomatic nuances.

Cautious use of GPT-4V is recommended due to hallucination risks.

Abstract

Generative AI, such as OpenAI's GPT-4V large-language model, has rapidly entered mainstream discourse. Novel capabilities in image processing and natural-language communication may augment existing forecasting methods. Large language models further display potential to better communicate weather hazards in a style honed for diverse communities and different languages. This study evaluates GPT-4V's ability to interpret meteorological charts and communicate weather hazards appropriately to the user, despite challenges of hallucinations, where generative AI delivers coherent, confident, but incorrect responses. We assess GPT-4V's competence via its web interface ChatGPT in two tasks: (1) generating a severe-weather outlook from weather-chart analysis and conducting self-evaluation, revealing an outlook that corresponds well with a Storm Prediction Center human-issued forecast; and (2) producing hazard summaries in Spanish and English from weather charts. Responses in Spanish, however, resemble direct (not idiomatic) translations from English to Spanish, yielding poorly translated summaries that lose critical idiomatic precision required for optimal communication. Our findings advocate for cautious integration of tools like GPT-4V in meteorology, underscoring the necessity of human oversight and development of trustworthy, explainable AI.