Exploring the Benefits of Teams in Multiagent Learning

TL;DR

This paper introduces a new multiagent team model inspired by organizational psychology, demonstrating that team-based reinforcement learning agents develop cooperation, better coordination, and higher rewards in social dilemmas.

Contribution

The paper proposes a novel multiagent team model for reinforcement learning inspired by organizational psychology, showing improved cooperation and coordination in social dilemmas.

Findings

Agents in teams develop cooperative policies.

Teams achieve higher rewards than non-cooperative setups.

Emergent role coordination improves team performance.

Abstract

For problems requiring cooperation, many multiagent systems implement solutions among either individual agents or across an entire population towards a common goal. Multiagent teams are primarily studied when in conflict; however, organizational psychology (OP) highlights the benefits of teams among human populations for learning how to coordinate and cooperate. In this paper, we propose a new model of multiagent teams for reinforcement learning (RL) agents inspired by OP and early work on teams in artificial intelligence. We validate our model using complex social dilemmas that are popular in recent multiagent RL and find that agents divided into teams develop cooperative pro-social policies despite incentives to not cooperate. Furthermore, agents are better able to coordinate and learn emergent roles within their teams and achieve higher rewards compared to when the interests of all agents are aligned.