Distributed Deep Reinforcement Learning Based Gradient Quantization for Federated Learning Enabled Vehicle Edge Computing

TL;DR

This paper proposes a distributed deep reinforcement learning approach to optimize gradient quantization in federated learning for vehicle edge computing, balancing training time and quantization error under dynamic channel conditions.

Contribution

It introduces a novel DRL-based scheme for joint optimization of quantization level and thresholds in FL-enabled VEC, addressing the challenge of time-varying channels.

Findings

The proposed scheme effectively reduces training time and quantization error.

Simulations demonstrate the scheme's superiority over baseline methods.

Optimal weighted factors between training time and QE are identified.

Abstract

Federated Learning (FL) can protect the privacy of the vehicles in vehicle edge computing (VEC) to a certain extent through sharing the gradients of vehicles' local models instead of local data. The gradients of vehicles' local models are usually large for the vehicular artificial intelligence (AI) applications, thus transmitting such large gradients would cause large per-round latency. Gradient quantization has been proposed as one effective approach to reduce the per-round latency in FL enabled VEC through compressing gradients and reducing the number of bits, i.e., the quantization level, to transmit gradients. The selection of quantization level and thresholds determines the quantization error, which further affects the model accuracy and training time. To do so, the total training time and quantization error (QE) become two key metrics for the FL enabled VEC. It is critical to jointly optimize the total training time and QE for the FL enabled VEC. However, the time-varying channel condition causes more challenges to solve this problem. In this paper, we propose a distributed deep reinforcement learning (DRL)-based quantization level allocation scheme to optimize the long-term reward in terms of the total training time and QE. Extensive simulations identify the optimal weighted factors between the total training time and QE, and demonstrate the feasibility and effectiveness of the proposed scheme.