Multiagent Cooperation and Competition with Deep Reinforcement Learning

TL;DR

This paper extends Deep Q-Learning to multiagent settings, demonstrating how independent agents can learn competitive or collaborative behaviors in Pong, revealing emergent strategies and the transition between behaviors.

Contribution

It introduces a multiagent Deep Q-Network framework and explores how different reward schemes lead to diverse emergent behaviors in a classic game.

Findings

Competitive agents learn to score efficiently.

Collaborative agents maximize game duration.

Behavior transitions from competition to cooperation.

Abstract

Multiagent systems appear in most social, economical, and political situations. In the present work we extend the Deep Q-Learning Network architecture proposed by Google DeepMind to multiagent environments and investigate how two agents controlled by independent Deep Q-Networks interact in the classic videogame Pong. By manipulating the classical rewarding scheme of Pong we demonstrate how competitive and collaborative behaviors emerge. Competitive agents learn to play and score efficiently. Agents trained under collaborative rewarding schemes find an optimal strategy to keep the ball in the game as long as possible. We also describe the progression from competitive to collaborative behavior. The present work demonstrates that Deep Q-Networks can become a practical tool for studying the decentralized learning of multiagent systems living in highly complex environments.