Development and Realization of Validation Benchmarks

TL;DR

This paper introduces a Bayesian validation framework that quantitatively assesses model accuracy by incorporating uncertainties, using probabilistic modeling and model reduction techniques, demonstrated on fluid flow in fractured porous media.

Contribution

It presents a novel Bayesian-based validation metric and a model reduction method for uncertainty-aware model validation in complex systems.

Findings

The framework effectively quantifies model validity considering uncertainties.

Bayesian Sparse Polynomial Chaos Expansion accelerates analysis of complex models.

Application to fractured porous media demonstrates practical utility.

Abstract

In the field of modeling, the word validation refers to simple comparisons between model outputs and experimental data. Usually, this comparison constitutes plotting the model results against data on the same axes to provide a visual assessment of agreement or lack thereof. However, there are a number of concerns with such naive comparisons. First, these comparisons tend to provide qualitative rather than quantitative assessments and are clearly insufficient for making decisions regarding model validity. Second, they often disregard or only partly account for existing uncertainties in the experimental observations or the model input parameters. Third, such comparisons can not reveal whether the model is appropriate for the intended purposes, as they mainly focus on the agreement in the observable quantities. These pitfalls give rise to the need for an uncertainty-aware framework that includes a validation metric. This metric shall provide a measure for comparison of the system response quantities of an experiment with the ones from a computational model, while accounting for uncertainties in both. To address this need, we have developed a statistical framework that incorporates a probabilistic modeling technique using a fully Bayesian approach. A Bayesian perspective yields an optimal bias-variance trade-off against the experimental data and provide an integrative metric for model validation that incorporates parameter and conceptual uncertainty. Additionally, to accelerate the analysis for computationally demanding flow and transport models in porous media, the framework is equipped with a model reduction technique, namely Bayesian Sparse Polynomial Chaos Expansion. We demonstrate the capabilities of the aforementioned Bayesian validation framework by applying it to an application for validation as well as uncertainty quantification of fluid flow in fractured porous media.