Neighborhood Cognition Consistent Multi-Agent Reinforcement Learning

TL;DR

This paper introduces neighborhood cognitive consistency into multi-agent reinforcement learning, enhancing cooperation in large-scale multi-agent systems through novel NCC-based algorithms demonstrated on various complex tasks.

Contribution

It proposes a general NCC framework for MARL, integrating cognitive consistency principles to improve cooperation, with specific implementations in deep Q-learning and Actor-Critic methods.

Findings

NCC-based methods outperform state-of-the-art MARL approaches.

The approach is effective in tasks like packet routing, wifi configuration, and football control.

NCC enhances large-scale multi-agent cooperation.

Abstract

Social psychology and real experiences show that cognitive consistency plays an important role to keep human society in order: if people have a more consistent cognition about their environments, they are more likely to achieve better cooperation. Meanwhile, only cognitive consistency within a neighborhood matters because humans only interact directly with their neighbors. Inspired by these observations, we take the first step to introduce \emph{neighborhood cognitive consistency} (NCC) into multi-agent reinforcement learning (MARL). Our NCC design is quite general and can be easily combined with existing MARL methods. As examples, we propose neighborhood cognition consistent deep Q-learning and Actor-Critic to facilitate large-scale multi-agent cooperations. Extensive experiments on several challenging tasks (i.e., packet routing, wifi configuration, and Google football player control) justify the superior performance of our methods compared with state-of-the-art MARL approaches.