# A multi-method study evaluating the inference of compartmental model parameters from a generative agent-based model

## TL;DR

This study compares optimization and Bayesian methods for calibrating SIR models using synthetic data generated by agent-based models to understand how contact structures affect parameter estimation.

## Contribution

The novel use of agent-based models to generate synthetic data for evaluating calibration methods in compartmental models.

## Key findings

- Nelder-Mead and HMC perform similarly in terms of accuracy metrics.
- HMC better captures ground truth parameters compared to Nelder-Mead.
- Effective infectious period is sensitive to contact patterns and susceptible population proportion.

## Abstract

Calibrating process models such as compartmental SIR Models to real data can be performed using either optimization or Bayesian techniques. To accurately assess the performance of these methods, synthetic outbreak data can be used. All information about the data generative process is known for synthetic data, while when using real data there are many unknowns such as under-reporting of cases or real parameter values. We propose using an agent-based model to generate synthetic data. Calibrating to synthetic datasets created using different agent contact structures can provide us with information on how changes in contact structures impact SIR model parameters. We compare results for two calibration methods: Nelder-Mead an optimization technique and HMC, a Bayesian technique. The analysis finds that the two calibration methods perform similar in terms of accuracy when looking at the Mean Absolute Error, Mean Absolute Scaled Error, and Relative Root Mean Squared Error. Looking at the model parameters, HMC is better able to capture the ground truth parameters then Nelder-Mead. The results of the calibration additionally show that the effective infectious period is sensitive to the changes in contact patterns and the proportion of susceptible individuals in the population. For choosing a calibration method, if overall accuracy is the desired outcome, either method should perform equally well, however, if the aim is to understand and analyse the model parameters HMC is a better choice. Understanding how the effective parameters such as the infectious period changes as contact patterns and vaccination rates change can provide valuable information in understanding how to interpret parameters calibrated from real world data that captures both isolation and vaccination.