Semantic Tracklets: An Object-Centric Representation for Visual Multi-Agent Reinforcement Learning

TL;DR

This paper introduces 'semantic tracklets', an object-centric representation that improves scalability and performance in visual multi-agent reinforcement learning tasks, enabling strategies for multiple agents from raw visual inputs.

Contribution

It proposes a novel object-centric representation called semantic tracklets that enhances scalability and learning efficiency in visual multi-agent reinforcement learning environments.

Findings

Semantic tracklets outperform baselines on VMPE.

Achieved +2.4 score difference over baselines on GFootball.

First to learn a five-player strategy in GFootball using only visual data.

Abstract

Solving complex real-world tasks, e.g., autonomous fleet control, often involves a coordinated team of multiple agents which learn strategies from visual inputs via reinforcement learning. Many existing multi-agent reinforcement learning (MARL) algorithms however don't scale to environments where agents operate on visual inputs. To address this issue, algorithmically, recent works have focused on non-stationarity and exploration. In contrast, we study whether scalability can also be achieved via a disentangled representation. For this, we explicitly construct an object-centric intermediate representation to characterize the states of an environment, which we refer to as `semantic tracklets.' We evaluate `semantic tracklets' on the visual multi-agent particle environment (VMPE) and on the challenging visual multi-agent GFootball environment. `Semantic tracklets' consistently outperform baselines on VMPE, and achieve a +2.4 higher score difference than baselines on GFootball. Notably, this method is the first to successfully learn a strategy for five players in the GFootball environment using only visual data.