An Algorithm For Adversary Aware Decentralized Networked MARL

TL;DR

This paper introduces an algorithm for decentralized multi-agent reinforcement learning that is resilient to adversarial agents, enabling consensus despite malicious deviations in a dynamic network.

Contribution

The work presents a novel algorithm that ensures consensus among non-adversarial agents in decentralized MARL with adversarial presence in dynamic networks.

Findings

Algorithm achieves consensus despite adversarial agents.

Resilience demonstrated in time-varying network settings.

Enhances security and robustness of decentralized MARL systems.

Abstract

Decentralized multi-agent reinforcement learning (MARL) algorithms have become popular in the literature since it allows heterogeneous agents to have their own reward functions as opposed to canonical multi-agent Markov Decision Process (MDP) settings which assume common reward functions over all agents. In this work, we follow the existing work on collaborative MARL where agents in a connected time varying network can exchange information among each other in order to reach a consensus. We introduce vulnerabilities in the consensus updates of existing MARL algorithms where agents can deviate from their usual consensus update, who we term as adversarial agents. We then proceed to provide an algorithm that allows non-adversarial agents to reach a consensus in the presence of adversaries under a constrained setting.