Stress-testing the coupled behavior of hybrid physics-machine learning climate simulations on an unseen, warmer climate

TL;DR

This paper evaluates the stability and accuracy of machine learning-based climate model parameterizations in hybrid simulations under unseen, warmer climate conditions, highlighting the importance of design choices and multi-climate training.

Contribution

It investigates how specific feature transformations and training strategies affect the out-of-distribution generalization of ML-based climate parameterizations in coupled simulations.

Findings

Design choices improve coupled performance but are insufficient alone for out-of-distribution generalization.

Training on multiple climate simulations enhances model robustness in new climate scenarios.

Certain feature transformations can inoculate models against non-physical extrapolation.

Abstract

Accurate and computationally-viable representations of clouds and turbulence are a long-standing challenge for climate model development. Traditional parameterizations that crudely but efficiently approximate these processes are a leading source of uncertainty in long-term projected warming and precipitation patterns. Machine Learning (ML)-based parameterizations have long been hailed as a promising alternative with the potential to yield higher accuracy at a fraction of the cost of more explicit simulations. However, these ML variants are often unpredictably unstable and inaccurate in \textit{coupled} testing (i.e. in a downstream hybrid simulation task where they are dynamically interacting with the large-scale climate model). These issues are exacerbated in out-of-distribution climates. Certain design decisions such as ``climate-invariant" feature transformation for moisture inputs, input vector expansion, and temporal history incorporation have been shown to improve coupled performance, but they may be insufficient for coupled out-of-distribution generalization. If feature selection and transformations can inoculate hybrid physics-ML climate models from non-physical, out-of-distribution extrapolation in a changing climate, there is far greater potential in extrapolating from observational data. Otherwise, training on multiple simulated climates becomes an inevitable necessity. While our results show generalization benefits from these design decisions, the obtained improvment does not sufficiently preclude the necessity of using multi-climate simulated training data.