Logical Team Q-learning: An approach towards factored policies in cooperative MARL

TL;DR

This paper introduces Logical Team Q-learning (LTQL), a novel method for learning factored policies in cooperative multi-agent reinforcement learning that is broadly applicable without environment assumptions.

Contribution

The paper presents LTQL, a new approach that enables factored policy learning in cooperative MARL without relying on environment-specific assumptions.

Findings

LTQL effectively learns factored policies in cooperative MARL scenarios.

Experimental results show LTQL's applicability in both tabular and deep learning settings.

LTQL outperforms baseline methods in collaborative tasks.

Abstract

We address the challenge of learning factored policies in cooperative MARL scenarios. In particular, we consider the situation in which a team of agents collaborates to optimize a common cost. The goal is to obtain factored policies that determine the individual behavior of each agent so that the resulting joint policy is optimal. The main contribution of this work is the introduction of Logical Team Q-learning (LTQL). LTQL does not rely on assumptions about the environment and hence is generally applicable to any collaborative MARL scenario. We derive LTQL as a stochastic approximation to a dynamic programming method we introduce in this work. We conclude the paper by providing experiments (both in the tabular and deep settings) that illustrate the claims.