Turn-based Multi-Agent Reinforcement Learning Model Checking

TL;DR

This paper introduces a scalable model checking approach for verifying turn-based multi-agent reinforcement learning agents in complex stochastic multiplayer games, addressing limitations of existing methods.

Contribution

It presents a novel integration of TMARL with model checking, improving verification scalability and effectiveness for large multi-agent environments.

Findings

Effective verification of TMARL agents demonstrated in various environments.

Scales better than naive monolithic model checking.

Applicable to complex stochastic multiplayer games.

Abstract

In this paper, we propose a novel approach for verifying the compliance of turn-based multi-agent reinforcement learning (TMARL) agents with complex requirements in stochastic multiplayer games. Our method overcomes the limitations of existing verification approaches, which are inadequate for dealing with TMARL agents and not scalable to large games with multiple agents. Our approach relies on tight integration of TMARL and a verification technique referred to as model checking. We demonstrate the effectiveness and scalability of our technique through experiments in different types of environments. Our experiments show that our method is suited to verify TMARL agents and scales better than naive monolithic model checking.