Diagnosis of model-structural errors with a sliding time-window Bayesian analysis

TL;DR

This paper introduces a Bayesian time-windowed analysis method to detect and diagnose time-dependent structural errors in hydrological models, enhancing model validation and improvement processes.

Contribution

The paper proposes a novel sliding time-window Bayesian approach using Bayesian model evidence to identify when models disqualify themselves due to structural errors over time.

Findings

Effective detection of model errors on various temporal scales

Application to soil moisture modeling demonstrates practical utility

Posterior parameter analysis aids in understanding error sources

Abstract

Deterministic hydrological models with uncertain, but inferred-to-be-time-invariant parameters typically show time-dependent model structural errors. Such errors can occur if a hydrological process is active in certain time periods in nature, but is not resolved by the model. Such missing processes could become visible during calibration as time-dependent best-fit values of model parameters. We propose a formal time-windowed Bayesian analysis to diagnose this type of model error, formalizing the question \In which period of the calibration time-series does the model statistically disqualify itself as quasi-true?" Using Bayesian model evidence (BME) as model performance metric, we determine how much the data in time windows of the calibration time-series support or refute the model. Then, we track BME over sliding time windows to obtain a dynamic, time-windowed BME (tBME) and search for sudden decreases that indicate an onset of model error. tBME also allows us to perform a formal, sliding likelihood-ratio test of the model against the data. Our proposed approach is designed to detect error occurrence on various temporal scales, which is especially useful in hydrological modelling. We illustrate this by applying our proposed method to soil moisture modeling. We test tBME as model error indicator on several synthetic and real-world test cases that we designed to vary in error sources and error time scales. Results prove the usefulness of the framework for detecting structural errors in dynamic models. Moreover, the time sequence of posterior parameter distributions helps to investigate the reasons for model error and provide guidance for model improvement.