Meta Federated Reinforcement Learning for Distributed Resource Allocation

TL;DR

This paper introduces a meta federated reinforcement learning framework for distributed resource allocation in cellular networks, improving energy efficiency and reducing transmission overhead by enabling local optimization and rapid adaptation.

Contribution

It proposes a novel MFRL framework that combines meta learning and federated reinforcement learning for efficient, distributed resource management in wireless networks.

Findings

Accelerates reinforcement learning convergence

Reduces transmission overhead

Outperforms conventional decentralized RL in energy efficiency

Abstract

In cellular networks, resource allocation is usually performed in a centralized way, which brings huge computation complexity to the base station (BS) and high transmission overhead. This paper explores a distributed resource allocation method that aims to maximize energy efficiency (EE) while ensuring the quality of service (QoS) for users. Specifically, in order to address wireless channel conditions, we propose a robust meta federated reinforcement learning (\textit{MFRL}) framework that allows local users to optimize transmit power and assign channels using locally trained neural network models, so as to offload computational burden from the cloud server to the local users, reducing transmission overhead associated with local channel state information. The BS performs the meta learning procedure to initialize a general global model, enabling rapid adaptation to different environments with improved EE performance. The federated learning technique, based on decentralized reinforcement learning, promotes collaboration and mutual benefits among users. Analysis and numerical results demonstrate that the proposed \textit{MFRL} framework accelerates the reinforcement learning process, decreases transmission overhead, and offloads computation, while outperforming the conventional decentralized reinforcement learning algorithm in terms of convergence speed and EE performance across various scenarios.