Verco: Learning Coordinated Verbal Communication for Multi-agent Reinforcement Learning

TL;DR

This paper introduces Verco, a multi-agent reinforcement learning framework that incorporates large language models to generate human-understandable communication, improving cooperation efficiency and interpretability in multi-agent systems.

Contribution

The paper presents a novel approach embedding large language models into multi-agent RL agents, enabling human-understandable communication and improved cooperative performance.

Findings

Enhanced learning efficiency in Overcooked game

Improved cooperative performance over existing methods

Provided interpretable communication mechanisms

Abstract

In recent years, multi-agent reinforcement learning algorithms have made significant advancements in diverse gaming environments, leading to increased interest in the broader application of such techniques. To address the prevalent challenge of partial observability, communication-based algorithms have improved cooperative performance through the sharing of numerical embedding between agents. However, the understanding of the formation of collaborative mechanisms is still very limited, making designing a human-understandable communication mechanism a valuable problem to address. In this paper, we propose a novel multi-agent reinforcement learning algorithm that embeds large language models into agents, endowing them with the ability to generate human-understandable verbal communication. The entire framework has a message module and an action module. The message module is responsible for generating and sending verbal messages to other agents, effectively enhancing information sharing among agents. To further enhance the message module, we employ a teacher model to generate message labels from the global view and update the student model through Supervised Fine-Tuning (SFT). The action module receives messages from other agents and selects actions based on current local observations and received messages. Experiments conducted on the Overcooked game demonstrate our method significantly enhances the learning efficiency and performance of existing methods, while also providing an interpretable tool for humans to understand the process of multi-agent cooperation.