Simulation Experiment Design for Calibration via Active Learning

TL;DR

This paper introduces two new active learning criteria for efficiently designing simulation experiments to calibrate complex models, improving the accuracy of posterior estimates and predictions with fewer simulation runs.

Contribution

It proposes novel sequential data acquisition criteria based on posterior uncertainty, enhancing simulation calibration efficiency and accuracy.

Findings

Improved posterior estimates in simulation calibration.

Enhanced prediction accuracy in epidemiological models.

More efficient use of simulation resources.

Abstract

Simulation models often have parameters as input and return outputs to understand the behavior of complex systems. Calibration is the process of estimating the values of the parameters in a simulation model in light of observed data from the system that is being simulated. When simulation models are expensive, emulators are built with simulation data as a computationally efficient approximation of an expensive model. An emulator then can be used to predict model outputs, instead of repeatedly running an expensive simulation model during the calibration process. Sequential design with an intelligent selection criterion can guide the process of collecting simulation data to build an emulator, making the calibration process more efficient and effective. This article proposes two novel criteria for sequentially acquiring new simulation data in an active learning setting by considering uncertainties on the posterior density of parameters. Analysis of several simulation experiments and real-data simulation experiments from epidemiology demonstrates that proposed approaches result in improved posterior and field predictions.