Promoting Cooperation in Multi-Agent Reinforcement Learning via Mutual Help

TL;DR

This paper introduces MH-MARL, a novel multi-agent reinforcement learning algorithm that enhances cooperation by enabling agents to predict and imitate each other's actions, leading to improved success rates and rewards.

Contribution

The paper proposes a new mutual-help mechanism in MARL that explicitly models and leverages agents' mutual influence to improve cooperative performance.

Findings

MH-MARL outperforms baseline algorithms in success rate.

MH-MARL achieves higher cumulative rewards.

Mutual imitation enhances coordination among agents.

Abstract

Multi-agent reinforcement learning (MARL) has achieved great progress in cooperative tasks in recent years. However, in the local reward scheme, where only local rewards for each agent are given without global rewards shared by all the agents, traditional MARL algorithms lack sufficient consideration of agents' mutual influence. In cooperative tasks, agents' mutual influence is especially important since agents are supposed to coordinate to achieve better performance. In this paper, we propose a novel algorithm Mutual-Help-based MARL (MH-MARL) to instruct agents to help each other in order to promote cooperation. MH-MARL utilizes an expected action module to generate expected other agents' actions for each particular agent. Then, the expected actions are delivered to other agents for selective imitation during training. Experimental results show that MH-MARL improves the performance of MARL both in success rate and cumulative reward.