Offsetting Unequal Competition through RL-assisted Incentive Schemes

TL;DR

This paper develops a novel RL-assisted incentive scheme to balance competition between unequal teams in a multi-agent environment, demonstrating improved performance of weaker teams through dynamic incentives.

Contribution

It introduces a new controller-assisted multi-agent reinforcement learning algorithm and a dynamic incentive scheme that adaptively balances rewards between competing teams.

Findings

Weak team performance improves with targeted incentives.

Incentive schemes can be optimized to promote learning in weaker agents.

Dynamic incentives lead to balanced team performance over time.

Abstract

This paper investigates the dynamics of competition among organizations with unequal expertise. Multi-agent reinforcement learning has been used to simulate and understand the impact of various incentive schemes designed to offset such inequality. We design Touch-Mark, a game based on well-known multi-agent-particle-environment, where two teams (weak, strong) with unequal but changing skill levels compete against each other. For training such a game, we propose a novel controller assisted multi-agent reinforcement learning algorithm \our\, which empowers each agent with an ensemble of policies along with a supervised controller that by selectively partitioning the sample space, triggers intelligent role division among the teammates. Using C-MADDPG as an underlying framework, we propose an incentive scheme for the weak team such that the final rewards of both teams become the same. We find that in spite of the incentive, the final reward of the weak team falls short of the strong team. On inspecting, we realize that an overall incentive scheme for the weak team does not incentivize the weaker agents within that team to learn and improve. To offset this, we now specially incentivize the weaker player to learn and as a result, observe that the weak team beyond an initial phase performs at par with the stronger team. The final goal of the paper has been to formulate a dynamic incentive scheme that continuously balances the reward of the two teams. This is achieved by devising an incentive scheme enriched with an RL agent which takes minimum information from the environment.