Explainable Offline-Online Training of Neural Networks for Parameterizations: A 1D Gravity Wave-QBO Testbed in the Small-data Regime

TL;DR

This paper demonstrates that online re-training of neural networks using ensemble Kalman inversion can produce realistic climate oscillations in a 1D gravity wave-QBO model, especially in small-data regimes, by focusing on filter-like kernel learning.

Contribution

It introduces an explainable offline-online training approach for neural network parameterizations in climate models, highlighting the effectiveness of partial re-training in small-data scenarios.

Findings

Offline training in big-data yields realistic QBOs.

Online re-training with ensemble Kalman inversion improves small-data performance.

Neural network kernels act as filters, capturing key dynamics.

Abstract

There are different strategies for training neural networks (NNs) as subgrid-scale parameterizations. Here, we use a 1D model of the quasi-biennial oscillation (QBO) and gravity wave (GW) parameterizations as testbeds. A 12-layer convolutional NN that predicts GW forcings for given wind profiles, when trained offline in a big-data regime (100-years), produces realistic QBOs once coupled to the 1D model. In contrast, offline training of this NN in a small-data regime (18-months) yields unrealistic QBOs. However, online re-training of just two layers of this NN using ensemble Kalman inversion and only time-averaged QBO statistics leads to parameterizations that yield realistic QBOs. Fourier analysis of these three NNs' kernels suggests why/how re-training works and reveals that these NNs primarily learn low-pass, high-pass, and a combination of band-pass filters, consistent with the importance of both local and non-local dynamics in GW propagation/dissipation. These findings/strategies apply to data-driven parameterizations of other climate processes generally.