Robustness Test for AI Forecasting of Hurricane Florence Using FourCastNetv2 and Random Perturbations of the Initial Condition

TL;DR

This study evaluates the robustness of NVIDIA's FourCastNetv2 AI weather model in forecasting Hurricane Florence by injecting various levels of noise into initial conditions and analyzing the impact on forecast accuracy and stability.

Contribution

The paper introduces a systematic method to test AI weather models' sensitivity to initial condition perturbations, demonstrating FourCastNetv2's robustness and limitations under noisy inputs.

Findings

FCNv2 preserves hurricane features under low to moderate noise.

Model maintains general storm trajectory despite high noise levels.

FCNv2 underestimates storm intensity and persistence consistently.

Abstract

Understanding the robustness of a weather forecasting model with respect to input noise or different uncertainties is important in assessing its output reliability, particularly for extreme weather events like hurricanes. In this paper, we test sensitivity and robustness of an artificial intelligence (AI) weather forecasting model: NVIDIAs FourCastNetv2 (FCNv2). We conduct two experiments designed to assess model output under different levels of injected noise in the models initial condition. First, we perturb the initial condition of Hurricane Florence from the European Centre for Medium-Range Weather Forecasts (ECMWF) Reanalysis v5 (ERA5) dataset (September 13-16, 2018) with varying amounts of Gaussian noise and examine the impact on predicted trajectories and forecasted storm intensity. Second, we start FCNv2 with fully random initial conditions and observe how the model responds to nonsensical inputs. Our results indicate that FCNv2 accurately preserves hurricane features under low to moderate noise injection. Even under high levels of noise, the model maintains the general storm trajectory and structure, although positional accuracy begins to degrade. FCNv2 consistently underestimates storm intensity and persistence across all levels of injected noise. With full random initial conditions, the model generates smooth and cohesive forecasts after a few timesteps, implying the models tendency towards stable, smoothed outputs. Our approach is simple and portable to other data-driven AI weather forecasting models.