Learning Communication Skills in Multi-task Multi-agent Deep Reinforcement Learning

TL;DR

This paper introduces Multi-task Communication Skills (MCS), a multi-agent deep reinforcement learning method that uses learnable communication protocols and a Transformer encoder to improve coordination across multiple tasks.

Contribution

The paper presents a novel multi-task MADRL framework with learnable communication protocols and a Transformer-based encoding, enhancing multi-task learning and agent coordination.

Findings

MCS outperforms baselines without communication.

MCS surpasses single-task MADRL methods.

Effective multi-task communication improves coordination.

Abstract

In multi-agent deep reinforcement learning (MADRL), agents can communicate with one another to perform a task in a coordinated manner. When multiple tasks are involved, agents can also leverage knowledge from one task to improve learning in other tasks. In this paper, we propose Multi-task Communication Skills (MCS), a MADRL with communication method that learns and performs multiple tasks simultaneously, with agents interacting through learnable communication protocols. MCS employs a Transformer encoder to encode task-specific observations into a shared message space, capturing shared communication skills among agents. To enhance coordination among agents, we introduce a prediction network that correlates messages with the actions of sender agents in each task. We adapt three multi-agent benchmark environments to multi-task settings, where the number of agents as well as the observation and action spaces vary across tasks. Experimental results demonstrate that MCS achieves better performance than multi-task MADRL baselines without communication, as well as single-task MADRL baselines with and without communication.