Calibrating Over-Parametrized Simulation Models: A Framework via Eligibility Set

TL;DR

This paper introduces a new calibration framework for complex, over-parametrized simulation models using eligibility sets, providing rigorous statistical guarantees and addressing challenges like non-identifiability.

Contribution

It proposes a set-based calibration method with a feature extraction approach that ensures statistical validity in high-dimensional, non-identifiable models.

Findings

Effective calibration of complex models demonstrated on numerical examples.

Framework provides rigorous frequentist guarantees.

Application to market simulator shows practical utility.

Abstract

Stochastic simulation aims to compute output performance for complex models that lack analytical tractability. To ensure accurate prediction, the model needs to be calibrated and validated against real data. Conventional methods approach these tasks by assessing the model-data match via simple hypothesis tests or distance minimization in an ad hoc fashion, but they can encounter challenges arising from non-identifiability and high dimensionality. In this paper, we investigate a framework to develop calibration schemes that satisfy rigorous frequentist statistical guarantees, via a basic notion that we call eligibility set designed to bypass non-identifiability via a set-based estimation. We investigate a feature extraction-then-aggregation approach to construct these sets that target at multivariate outputs. We demonstrate our methodology on several numerical examples, including an application to calibration of a limit order book market simulator (ABIDES).