Decentralized Coordination in Partially Observable Queueing Networks

TL;DR

This paper introduces an attention-based communication model for multi-agent queueing networks with partial observations, enabling agents to efficiently share information, reduce packet drops, and improve routing policies.

Contribution

It presents ATVC, a novel attention-based communication method that selectively shares information among agents, enhancing coordination in partially observable queueing systems.

Findings

ATVC effectively infers true queue states.

The approach reduces packet drop rates.

It outperforms existing baseline methods.

Abstract

We consider communication in a fully cooperative multi-agent system, where the agents have partial observation of the environment and must act jointly to maximize the overall reward. We have a discrete-time queueing network where agents route packets to queues based only on the partial information of the current queue lengths. The queues have limited buffer capacity, so packet drops happen when they are sent to a full queue. In this work, we implemented a communication channel for the agents to share their information in order to reduce the packet drop rate. For efficient information sharing we use an attention-based communication model, called ATVC, to select informative messages from other agents. The agents then infer the state of queues using a combination of the variational auto-encoder, VAE, and product-of-experts, PoE, model. Ultimately, the agents learn what they need to communicate and with whom, instead of communicating all the time with everyone. We also show empirically that ATVC is able to infer the true state of the queues and leads to a policy which outperforms existing baselines.