FREIDA: A Framework for developing quantitative agent based models based on qualitative expert knowledge

TL;DR

FREIDA is a systematic framework that integrates qualitative expert knowledge with quantitative data to develop, calibrate, and validate agent-based models, especially useful in data-sparse contexts.

Contribution

It introduces a novel method to extract Expected System Behaviors from qualitative data, enabling comprehensive model development and validation.

Findings

Effective incorporation of qualitative insights into ABMs

Quantitative scoring of model behaviors using ESBs

Successful case study on criminal networks in the Netherlands

Abstract

Agent Based Models (ABMs) often deal with systems where there is a lack of quantitative data or where quantitative data alone may be insufficient to fully capture the complexities of real-world systems. Expert knowledge and qualitative insights, such as those obtained through interviews, ethnographic research, historical accounts, or participatory workshops, are critical in constructing realistic behavioral rules, interactions, and decision-making processes within these models. However, there is a lack of systematic approaches that are able to incorporate both qualitative and quantitative data across the entire modeling cycle. To address this, we propose FREIDA (FRamework for Expert-Informed Data-driven Agent-based models), a systematic mixed-methods framework to develop, train, and validate ABMs, particularly in data-sparse contexts. Our main technical innovation is to extract what we call Expected System Behaviors (ESBs) from qualitative data, which are testable statements that can be evaluated on model simulations. Divided into Calibration Statements (CS) for model calibration and Validation Statements (VS) for model validation, they provide a quantitative scoring mechanism on the same footing as quantitative data. In this way, qualitative insights can inform not only model specification but also its parameterization and assessment of fitness for purpose, which is a long standing challenge. We illustrate the application of FREIDA through a case study of criminal cocaine networks in the Netherlands.