A Novel Multiple Access Scheme for Heterogeneous Wireless Communications using Symmetry-aware Continual Deep Reinforcement Learning

TL;DR

This paper introduces a symmetry-aware continual deep reinforcement learning approach for adaptive multiple access in heterogeneous wireless environments, improving throughput, fairness, and responsiveness for Metaverse applications.

Contribution

It proposes a novel symmetry-aware continual learning mechanism integrated with D3QL for MAC protocols, enabling backward compatibility and adaptability in dynamic, heterogeneous wireless settings.

Findings

Outperforms traditional DRL methods in throughput and fairness

Reduces collision rates in dynamic environments

Ensures real-time responsiveness and adaptability

Abstract

The Metaverse holds the potential to revolutionize digital interactions through the establishment of a highly dynamic and immersive virtual realm over wireless communications systems, offering services such as massive twinning and telepresence. This landscape presents novel challenges, particularly efficient management of multiple access to the frequency spectrum, for which numerous adaptive Deep Reinforcement Learning (DRL) approaches have been explored. However, challenges persist in adapting agents to heterogeneous and non-stationary wireless environments. In this paper, we present a novel approach that leverages Continual Learning (CL) to enhance intelligent Medium Access Control (MAC) protocols, featuring an intelligent agent coexisting with legacy User Equipments (UEs) with varying numbers, protocols, and transmission profiles unknown to the agent for the sake of backward compatibility and privacy. We introduce an adaptive Double and Dueling Deep Q-Learning (D3QL)-based MAC protocol, enriched by a symmetry-aware CL mechanism, which maximizes intelligent agent throughput while ensuring fairness. Mathematical analysis validates the efficiency of our proposed scheme, showcasing superiority over conventional DRL-based techniques in terms of throughput, collision rate, and fairness, coupled with real-time responsiveness in highly dynamic scenarios.