VIL2C: Value-of-Information Aware Low-Latency Communication for Multi-Agent Reinforcement Learning

TL;DR

This paper introduces VIL2C, a communication scheme for multi-agent reinforcement learning that dynamically prioritizes important messages and adapts reception timing to reduce latency and improve performance in time-sensitive tasks.

Contribution

The paper proposes a novel VOI-based low-latency communication method with adaptive reception, providing theoretical performance guarantees and demonstrating superior results over existing approaches.

Findings

VIL2C improves MARL performance under various communication conditions.

The scheme effectively prioritizes high-VOI messages for low-latency transmission.

Adaptive reception reduces unnecessary waiting, enhancing system responsiveness.

Abstract

Inter-agent communication serves as an effective mechanism for enhancing performance in collaborative multi-agent reinforcement learning(MARL) systems. However, the inherent communication latency in practical systems induces both action decision delays and outdated information sharing, impeding MARL performance gains, particularly in time-critical applications like autonomous driving. In this work, we propose a Value-of-Information aware Low-latency Communication(VIL2C) scheme that proactively adjusts the latency distribution to mitigate its effects in MARL systems. Specifically, we define a Value of Information (VOI) metric to quantify the importance of delayed message transmission based on each delayed message's importance. Moreover, we propose a progressive message reception mechanism to adaptively adjust the reception duration based on received messages. We derive the optimized VoI aware resource allocation and theoretically prove the performance advantage of the proposed VIL2C scheme. Extensive experiments demonstrate that VIL2C outperforms existing approaches under various communication conditions. These gains are attributed to the low-latency transmission of high-VoI messages via resource allocation and the elimination of unnecessary waiting periods via adaptive reception duration.