Q-Mixing Network for Multi-Agent Pathfinding in Partially Observable Grid Environments

TL;DR

This paper introduces a Q-mixing network approach for multi-agent pathfinding in partially observable grid environments, enabling scalable cooperative navigation through reinforcement learning.

Contribution

It proposes a novel mixing Q-network architecture that improves cooperative multi-agent pathfinding in partially observable settings.

Findings

Approach scales well with the number of agents.

Agents learn effective cooperative policies.

Method outperforms baseline approaches in experiments.

Abstract

In this paper, we consider the problem of multi-agent navigation in partially observable grid environments. This problem is challenging for centralized planning approaches as they, typically, rely on the full knowledge of the environment. We suggest utilizing the reinforcement learning approach when the agents, first, learn the policies that map observations to actions and then follow these policies to reach their goals. To tackle the challenge associated with learning cooperative behavior, i.e. in many cases agents need to yield to each other to accomplish a mission, we use a mixing Q-network that complements learning individual policies. In the experimental evaluation, we show that such approach leads to plausible results and scales well to large number of agents.