HECT: High-Dimensional Ensemble Consistency Testing for Climate Models

TL;DR

This paper introduces HECT, a novel high-dimensional ensemble consistency testing method for climate models, utilizing probabilistic classifiers to assess the statistical reproducibility of complex spatio-temporal simulation outputs.

Contribution

It develops a new statistical testing framework employing machine learning classifiers to evaluate the consistency of high-dimensional climate model outputs.

Findings

Effective detection of model inconsistencies

Applicable to complex spatio-temporal data

Enhances model validation processes

Abstract

Climate models play a crucial role in understanding the effect of environmental and man-made changes on climate to help mitigate climate risks and inform governmental decisions. Large global climate models such as the Community Earth System Model (CESM), developed by the National Center for Atmospheric Research, are very complex with millions of lines of code describing interactions of the atmosphere, land, oceans, and ice, among other components. As development of the CESM is constantly ongoing, simulation outputs need to be continuously controlled for quality. To be able to distinguish a "climate-changing" modification of the code base from a true climate-changing physical process or intervention, there needs to be a principled way of assessing statistical reproducibility that can handle both spatial and temporal high-dimensional simulation outputs. Our proposed work uses probabilistic classifiers like tree-based algorithms and deep neural networks to perform a statistically rigorous goodness-of-fit test of high-dimensional spatio-temporal data.