Learning to Learn Group Alignment: A Self-Tuning Credo Framework with Multiagent Teams

TL;DR

This paper introduces a self-tuning framework for multiagent teams where agents dynamically adjust their incentives and group alignment during learning, leading to improved collective outcomes.

Contribution

It extends hierarchical reinforcement learning by enabling agents to self-regulate their incentive structures and adapt group alignments dynamically during training.

Findings

Agents achieve better global outcomes with self-tuning.

Dynamic group alignment improves multiagent cooperation.

Preliminary results show promising performance gains.

Abstract

Mixed incentives among a population with multiagent teams has been shown to have advantages over a fully cooperative system; however, discovering the best mixture of incentives or team structure is a difficult and dynamic problem. We propose a framework where individual learning agents self-regulate their configuration of incentives through various parts of their reward function. This work extends previous work by giving agents the ability to dynamically update their group alignment during learning and by allowing teammates to have different group alignment. Our model builds on ideas from hierarchical reinforcement learning and meta-learning to learn the configuration of a reward function that supports the development of a behavioral policy. We provide preliminary results in a commonly studied multiagent environment and find that agents can achieve better global outcomes by self-tuning their respective group alignment parameters.