Signal attenuation enables scalable decentralized multi-agent reinforcement learning over networks

TL;DR

This paper demonstrates that signal attenuation in wireless networks can facilitate scalable decentralized multi-agent reinforcement learning by enabling local observations, with applications shown in radar target detection and power allocation.

Contribution

It introduces new decentralized MARL formulations leveraging signal decay, derives local approximations with error bounds, and develops saddle point policy gradient algorithms for wireless network problems.

Findings

Signal attenuation enables decentralization in MARL.

Proposed algorithms effectively solve power allocation in radar networks.

The approach offers a model extendable to other wireless communication problems.

Abstract

Multi-agent reinforcement learning (MARL) methods typically require that agents enjoy global state observability, preventing development of decentralized algorithms and limiting scalability. Recent work has shown that, under assumptions on decaying inter-agent influence, global observability can be replaced by local neighborhood observability at each agent, enabling decentralization and scalability. Real-world applications enjoying such decay properties remain underexplored, however, despite the fact that signal power decay, or signal attenuation, due to path loss is an intrinsic feature of many problems in wireless communications and radar networks. In this paper, we show that signal attenuation enables decentralization in MARL by considering the illustrative special case of performing power allocation for target detection in a radar network. To achieve this, we propose two new constrained multi-agent Markov decision process formulations of this power allocation problem, derive local neighborhood approximations for global value function and policy gradient estimates and establish corresponding error bounds, and develop decentralized saddle point policy gradient algorithms for solving the proposed problems. Our approach, though oriented towards the specific radar network problem we consider, provides a useful model for extensions to additional problems in wireless communications and radar networks.