Rescale-Invariant Federated Reinforcement Learning for Resource Allocation in V2X Networks

TL;DR

This paper introduces a rescale-invariant federated reinforcement learning algorithm that leverages ReLU neural network properties to improve resource allocation in V2X networks, achieving faster and better convergence.

Contribution

It proposes a novel rescale-invariant FRL method based on node-wise invariance of ReLU networks, addressing data discrepancy issues in V2X resource allocation.

Findings

Enhanced convergence speed in simulations

Improved learning performance over existing methods

Effective handling of data discrepancy among agents

Abstract

Federated Reinforcement Learning (FRL) offers a promising solution to various practical challenges in resource allocation for vehicle-to-everything (V2X) networks. However, the data discrepancy among individual agents can significantly degrade the performance of FRL-based algorithms. To address this limitation, we exploit the node-wise invariance property of ReLU-activated neural networks, with the aim of reducing data discrepancy to improve learning performance. Based on this property, we introduce a backward rescale-invariant operation to develop a rescale-invariant FRL algorithm. Simulation results demonstrate that the proposed algorithm notably enhances both convergence speed and convergent performance.