All by Myself: Learning Individualized Competitive Behaviour with a Contrastive Reinforcement Learning optimization

TL;DR

This paper introduces a contrastive reinforcement learning model that enables agents to develop personalized strategies in competitive multiplayer games, adapting to specific opponents and improving performance through online learning.

Contribution

The paper presents a novel neural network architecture and training method for learning individualized strategies in competitive multi-agent environments, addressing limitations of prior centralized or continual learning approaches.

Findings

Model outperforms baseline strategies in multiple game scenarios.

Effective in learning opponent-specific tactics over repeated interactions.

Demonstrates adaptability to both offline and online competitive settings.

Abstract

In a competitive game scenario, a set of agents have to learn decisions that maximize their goals and minimize their adversaries' goals at the same time. Besides dealing with the increased dynamics of the scenarios due to the opponents' actions, they usually have to understand how to overcome the opponent's strategies. Most of the common solutions, usually based on continual learning or centralized multi-agent experiences, however, do not allow the development of personalized strategies to face individual opponents. In this paper, we propose a novel model composed of three neural layers that learn a representation of a competitive game, learn how to map the strategy of specific opponents, and how to disrupt them. The entire model is trained online, using a composed loss based on a contrastive optimization, to learn competitive and multiplayer games. We evaluate our model on a pokemon duel scenario and the four-player competitive Chef's Hat card game. Our experiments demonstrate that our model achieves better performance when playing against offline, online, and competitive-specific models, in particular when playing against the same opponent multiple times. We also present a discussion on the impact of our model, in particular on how well it deals with on specific strategy learning for each of the two scenarios.