Green Deep Reinforcement Learning for Radio Resource Management: Architecture, Algorithm Compression and Challenge

TL;DR

This paper explores innovative architecture and algorithmic strategies to achieve energy-efficient deep reinforcement learning for radio resource management in 5G networks, addressing environmental concerns while maintaining high performance.

Contribution

It introduces a cloud-based distributed DRL architecture and compression techniques for neural networks and MDPs, enhancing green RRM capabilities.

Findings

Proposed a cloud-based training and decision-making scheme for RRM.

Introduced compression methods for neural networks and MDPs.

Developed a spatial transfer learning scheme to improve efficiency.

Abstract

AI heralds a step-change in the performance and capability of wireless networks and other critical infrastructures. However, it may also cause irreversible environmental damage due to their high energy consumption. Here, we address this challenge in the context of 5G and beyond, where there is a complexity explosion in radio resource management (RRM). On the one hand, deep reinforcement learning (DRL) provides a powerful tool for scalable optimization for high dimensional RRM problems in a dynamic environment. On the other hand, DRL algorithms consume a high amount of energy over time and risk compromising progress made in green radio research. This paper reviews and analyzes how to achieve green DRL for RRM via both architecture and algorithm innovations. Architecturally, a cloud based training and distributed decision-making DRL scheme is proposed, where RRM entities can make lightweight deep local decisions whilst assisted by on-cloud training and updating. On the algorithm level, compression approaches are introduced for both deep neural networks and the underlying Markov Decision Processes, enabling accurate low-dimensional representations of challenges. To scale learning across geographic areas, a spatial transfer learning scheme is proposed to further promote the learning efficiency of distributed DRL entities by exploiting the traffic demand correlations. Together, our proposed architecture and algorithms provide a vision for green and on-demand DRL capability.