Another Look at Statistical Calibration: A Non-Asymptotic Theory and Prediction-Oriented Optimality

TL;DR

This paper develops a non-asymptotic, prediction-focused calibration framework for computer models, addressing practical issues of model inadequacy and limited data, with theoretical guarantees and empirical validation.

Contribution

It introduces a novel non-asymptotic calibration method that minimizes predictive error and connects to Bayesian approaches, with proven statistical guarantees.

Findings

The proposed method achieves better predictive accuracy in synthetic and real data.

The calibration approach provides finite-sample guarantees.

Connections to Bayesian calibration enhance interpretability and flexibility.

Abstract

We provide another look at the statistical calibration problem in computer models. This viewpoint is inspired by two overarching practical considerations of computer models: (i) many computer models are inadequate for perfectly modeling physical systems, even with the best-tuned calibration parameters; (ii) only a finite number of data points are available from the physical experiment associated with a computer model. Following this new line of thinking, we provide a non-asymptotic theory and derive a prediction-oriented calibration method. Our calibration method minimizes the predictive mean squared error for a finite sample size with statistical guarantees. We introduce an algorithm to perform the proposed calibration method and connect it to existing Bayesian calibration methods. Synthetic and real examples are provided to corroborate the derived theory and illustrate some advantages of the proposed calibration method.