Domain-driven Metrics for Reinforcement Learning: A Case Study on Epidemic Control using Agent-based Simulation

TL;DR

This paper introduces domain-driven metrics for reinforcement learning applied to agent-based epidemic models, demonstrating their effectiveness in evaluating policies like masking, vaccination, and lockdown in pandemic simulations.

Contribution

It develops and applies domain-specific metrics for RL in complex, stochastic agent-based models, enhancing performance assessment in epidemic control scenarios.

Findings

Domain-driven rewards improve policy evaluation.

Metrics effectively differentiate simulation scenarios.

RL policies show varied effectiveness based on domain metrics.

Abstract

For the development and optimization of agent-based models (ABMs) and rational agent-based models (RABMs), optimization algorithms such as reinforcement learning are extensively used. However, assessing the performance of RL-based ABMs and RABMS models is challenging due to the complexity and stochasticity of the modeled systems, and the lack of well-standardized metrics for comparing RL algorithms. In this study, we are developing domain-driven metrics for RL, while building on state-of-the-art metrics. We demonstrate our ``Domain-driven-RL-metrics'' using policy optimization on a rational ABM disease modeling case study to model masking behavior, vaccination, and lockdown in a pandemic. Our results show the use of domain-driven rewards in conjunction with traditional and state-of-the-art metrics for a few different simulation scenarios such as the differential availability of masks.