Active Legibility in Multiagent Reinforcement Learning

TL;DR

This paper introduces a multiagent reinforcement learning framework focused on active legibility, enabling agents to reveal intentions through behavior to improve collaboration and efficiency in complex decision-making scenarios.

Contribution

It proposes a novel active legibility framework for multiagent RL, emphasizing behavior-based intention signaling to enhance agent cooperation and reduce training time.

Findings

The framework improves efficiency over existing algorithms.

Agents demonstrate better collaboration through legible actions.

Training time is significantly reduced.

Abstract

A multiagent sequential decision problem has been seen in many critical applications including urban transportation, autonomous driving cars, military operations, etc. Its widely known solution, namely multiagent reinforcement learning, has evolved tremendously in recent years. Among them, the solution paradigm of modeling other agents attracts our interest, which is different from traditional value decomposition or communication mechanisms. It enables agents to understand and anticipate others' behaviors and facilitates their collaboration. Inspired by recent research on the legibility that allows agents to reveal their intentions through their behavior, we propose a multiagent active legibility framework to improve their performance. The legibility-oriented framework allows agents to conduct legible actions so as to help others optimise their behaviors. In addition, we design a series of problem domains that emulate a common scenario and best characterize the legibility in multiagent reinforcement learning. The experimental results demonstrate that the new framework is more efficient and costs less training time compared to several multiagent reinforcement learning algorithms.