Improving Model Performance by Adapting the KGE Metric to Account for System Non-Stationarity

TL;DR

The paper introduces the JKGE_ss metric, which detects and accounts for non-stationarity in geoscientific data, leading to improved model performance across diverse hydroclimatic conditions.

Contribution

A novel metric, JKGE_ss, is proposed to better evaluate models by incorporating system non-stationarity, unlike traditional metrics.

Findings

JKGE_ss improves reproduction of system temporal dynamics.

Enhanced model performance across wet, dry, and recession periods.

Traditional metrics may mislead model assessment under changing conditions.

Abstract

Geoscientific systems tend to be characterized by pronounced temporal non-stationarity, arising from seasonal and climatic variability in hydrometeorological drivers, and from natural and anthropogenic changes to land use and cover. As has been pointed out, such variability renders "the assumption of statistical stationarity obsolete in water management", and requires us to "account for, rather than ignore, non-stationary trends" in the data. However, metrics used for model development are typically based on the implicit and unjustifiable assumption that the data generating process is time-stationary. Here, we introduce the JKGE_ss metric (adapted from KGE_ss) that detects and accounts for dynamical non-stationarity in the statistical properties of the data and thereby improves information extraction and model performance. Unlike NSE and KGE_ss, which use the long-term mean as a benchmark against which to evaluate model efficiency, JKGE_ss emphasizes reproduction of temporal variations in system storage. We tested the robustness of the new metric by training physical-conceptual and data-based catchment-scale models of varying complexity across a wide range of hydroclimatic conditions, from recent-precipitation-dominated to snow-dominated to strongly arid. In all cases, the result was improved reproduction of system temporal dynamics at all time scales, across wet to dry years, and over the full range of flow levels (especially recession periods). Since traditional metrics fail to adequately account for temporal shifts in system dynamics, potentially resulting in misleading assessments of model performance under changing conditions, we recommend the adoption of JKGE_ss for geoscientific model development.