Semantic-Aware Collaborative Deep Reinforcement Learning Over Wireless Cellular Networks

TL;DR

This paper introduces a semantic-aware collaborative deep reinforcement learning framework that enables heterogeneous agents to efficiently collaborate over wireless networks by selecting relevant agents and jointly optimizing training and bandwidth allocation.

Contribution

It proposes a novel semantic-aware federated DRL algorithm that improves collaboration efficiency among heterogeneous agents in resource-limited wireless environments.

Findings

Outperforms state-of-the-art baselines in simulations.

Effectively selects semantically relevant agents for collaboration.

Optimizes training and bandwidth allocation for real-time tasks.

Abstract

Collaborative deep reinforcement learning (CDRL) algorithms in which multiple agents can coordinate over a wireless network is a promising approach to enable future intelligent and autonomous systems that rely on real-time decision-making in complex dynamic environments. Nonetheless, in practical scenarios, CDRL faces many challenges due to the heterogeneity of agents and their learning tasks, different environments, time constraints of the learning, and resource limitations of wireless networks. To address these challenges, in this paper, a novel semantic-aware CDRL method is proposed to enable a group of heterogeneous untrained agents with semantically-linked DRL tasks to collaborate efficiently across a resource-constrained wireless cellular network. To this end, a new heterogeneous federated DRL (HFDRL) algorithm is proposed to select the best subset of semantically relevant DRL agents for collaboration. The proposed approach then jointly optimizes the training loss and wireless bandwidth allocation for the cooperating selected agents in order to train each agent within the time limit of its real-time task. Simulation results show the superior performance of the proposed algorithm compared to state-of-the-art baselines.