Models we Can Trust: Toward a Systematic Discipline of (Agent-Based) Model Interpretation and Validation

TL;DR

This paper advocates for establishing a systematic discipline for interpreting and validating social science models, emphasizing formal frameworks, behavioral equivalence, and robustness analysis to enhance trustworthiness.

Contribution

It proposes new logical and theoretical tools for formal specification, behavioral comparison, and robustness assessment of agent-based and mathematical social models.

Findings

Introduces formal frameworks for specifying social mechanisms.

Adapts reactive system tools like bisimulation for model comparison.

Develops an adversarial theory to test model robustness.

Abstract

We advocate the development of a discipline of interacting with and extracting information from models, both mathematical (e.g. game-theoretic ones) and computational (e.g. agent-based models). We outline some directions for the development of a such a discipline: - the development of logical frameworks for the systematic formal specification of stylized facts and social mechanisms in (mathematical and computational) social science. Such frameworks would bring to attention new issues, such as phase transitions, i.e. dramatical changes in the validity of the stylized facts beyond some critical values in parameter space. We argue that such statements are useful for those logical frameworks describing properties of ABM. - the adaptation of tools from the theory of reactive systems (such as bisimulation) to obtain practically relevant notions of two systems "having the same behavior". - the systematic development of an adversarial theory of model perturbations, that investigates the robustness of conclusions derived from models of social behavior to variations in several features of the social dynamics. These may include: activation order, the underlying social network, individual agent behavior.