Emergent Cooperation in Quantum Multi-Agent Reinforcement Learning Using Communication

TL;DR

This paper explores how communication protocols can foster emergent cooperation in quantum multi-agent reinforcement learning, demonstrating high cooperation levels across social dilemmas.

Contribution

It introduces and evaluates communication-based methods for quantum multi-agent reinforcement learning, a novel extension of classical approaches.

Findings

Communication protocols like MATE and MEDIATE achieve high cooperation levels.

Quantum multi-agent RL can effectively utilize communication for cooperation.

Experimental results validate the viability of communication in quantum settings.

Abstract

Emergent cooperation in classical Multi-Agent Reinforcement Learning has gained significant attention, particularly in the context of Sequential Social Dilemmas (SSDs). While classical reinforcement learning approaches have demonstrated capability for emergent cooperation, research on extending these methods to Quantum Multi-Agent Reinforcement Learning remains limited, particularly through communication. In this paper, we apply communication approaches to quantum Q-Learning agents: the Mutual Acknowledgment Token Exchange (MATE) protocol, its extension Mutually Endorsed Distributed Incentive Acknowledgment Token Exchange (MEDIATE), the peer rewarding mechanism Gifting, and Reinforced Inter-Agent Learning (RIAL). We evaluate these approaches in three SSDs: the Iterated Prisoner's Dilemma, Iterated Stag Hunt, and Iterated Game of Chicken. Our experimental results show that approaches using MATE with temporal-difference measure (MATE\textsubscript{TD}), AutoMATE, MEDIATE-I, and MEDIATE-S achieved high cooperation levels across all dilemmas, demonstrating that communication is a viable mechanism for fostering emergent cooperation in Quantum Multi-Agent Reinforcement Learning.