Test case generation for agent-based models: A systematic literature review

TL;DR

This systematic review examines test case generation methods for agent-based models, highlighting current techniques' strengths and gaps, especially in testing society-level behaviors and the need for realistic case studies.

Contribution

It provides a comprehensive overview of test case generation in agent-based models, identifying research gaps and suggesting directions for future work.

Findings

Most techniques effectively test functional requirements at agent and integration levels

Few techniques are capable of testing society-level behavior

A need exists for more realistic case studies in evaluation

Abstract

Agent-based models play an important role in simulating complex emergent phenomena and supporting critical decisions. In this context, a software fault may result in poorly informed decisions that lead to disastrous consequences. The ability to rigorously test these models is therefore essential. In this systematic literature review, we answer five research questions related to the key aspects of test case generation in agent-based models: What are the information artifacts used to generate tests? How are these tests generated? How is a verdict assigned to a generated test? How is the adequacy of a generated test suite measured? What level of abstraction of an agent-based model is targeted by a generated test? Our results show that whilst the majority of techniques are effective for testing functional requirements at the agent and integration levels of abstraction, there are comparatively few techniques capable of testing society-level behaviour. Additionally, we identify a need for more thorough evaluation using realistic case studies that feature challenging properties associated with a typical agent-based model.