Interaction-Aware Multi-Agent Reinforcement Learning for Mobile Agents with Individual Goals

TL;DR

This paper introduces a curriculum-based multi-agent reinforcement learning approach for mobile agents with individual goals, addressing non-stationarity and interaction challenges in decentralized multi-agent systems.

Contribution

It proposes a two-stage curriculum strategy enabling agents to learn goal achievement and interaction policies in non-stationary multi-agent environments.

Findings

Effective in autonomous driving lane-change scenarios

Improves multi-agent interaction learning

Enhances decentralized policy training

Abstract

In a multi-agent setting, the optimal policy of a single agent is largely dependent on the behavior of other agents. We investigate the problem of multi-agent reinforcement learning, focusing on decentralized learning in non-stationary domains for mobile robot navigation. We identify a cause for the difficulty in training non-stationary policies: mutual adaptation to sub-optimal behaviors, and we use this to motivate a curriculum-based strategy for learning interactive policies. The curriculum has two stages. First, the agent leverages policy gradient algorithms to learn a policy that is capable of achieving multiple goals. Second, the agent learns a modifier policy to learn how to interact with other agents in a multi-agent setting. We evaluated our approach on both an autonomous driving lane-change domain and a robot navigation domain.