Surrogate-based Bayesian calibration methods for chaotic systems: a comparison of traditional and non-traditional approaches

TL;DR

This paper compares four emulator-based Bayesian calibration methods for chaotic systems, highlighting their performance, practical considerations, and the effectiveness of goal-oriented approaches in reducing computational costs.

Contribution

It introduces and evaluates a goal-oriented Bayesian calibration extension (GBOED), demonstrating its advantages over traditional methods in complex dynamical systems.

Findings

GBOED aligns design with inference, improving calibration efficiency.

CES, HM, and GBOED perform well with limited evaluations.

Standard BOED underperforms in chaotic system calibration.

Abstract

Parameter calibration is essential for reducing uncertainty and improving predictive fidelity in physics-based models, yet it is often limited by the high computational cost of model evaluations. Bayesian calibration methods provide a principled framework for combining prior information with data while rigorously quantifying uncertainty. In this work, we compare four emulator-based Bayesian calibration strategies, Calibrate-Emulate-Sample (CES), History Matching (HM), Bayesian Optimal Experimental Design (BOED), and a goal-oriented extension of BOED (GBOED). The proposed GBOED formulation explicitly targets information gain with respect to the calibration posterior, aligning design decisions with downstream inference. We assess methods using accuracy and uncertainty quantification metrics, convergence behavior under increasing computational budgets, and practical considerations such as implementation complexity and robustness. For the Lorenz '96 system, CES, HM, and GBOED all yield strong calibration performance, even with limited numbers of model evaluations, while standard BOED generally underperforms in this setting. Differences among the strongest methods are modest, particularly as computational budgets increase. For the two-layer quasi-geostrophic system, all methods produce reasonable posterior estimates, and convergence behavior is more consistent. Overall, our results indicate that multiple emulator-based calibration strategies can perform comparably well when applied appropriately, with method selection often guided more by computational and practical considerations than by accuracy alone. These findings highlight both the limitations of standard BOED for calibration and the promise of goal-oriented and iterative approaches for efficient Bayesian inference in complex dynamical systems.