Federated Learning With Energy Harvesting Devices: An MDP Framework

TL;DR

This paper proposes an MDP-based framework for federated learning with energy harvesting devices, optimizing device scheduling and power control to improve convergence and energy efficiency.

Contribution

It introduces a novel MDP model for energy-aware device scheduling in federated learning and develops low-complexity and deep reinforcement learning algorithms leveraging policy structure.

Findings

Optimal transmission policy has a monotone structure.

Proposed algorithms are asymptotically optimal with many devices.

Numerical results validate the effectiveness of the algorithms.

Abstract

Federated learning (FL) necessitates that edge devices conduct local training and communicate with a parameter server, resulting in significant energy consumption. A key challenge in practical FL systems is the rapid depletion of battery-limited edge devices, which limits their operational lifespan and impacts learning performance. To tackle this issue, we implement energy harvesting techniques in FL systems to capture ambient energy, thereby providing continuous power to edge devices. We first establish the convergence bound for the wireless FL system with energy harvesting devices, illustrating that the convergence is affected by partial device participation and packet drops, both of which depend on the energy supply. To accelerate the convergence, we formulate a joint device scheduling and power control problem and model it as a Markov decision process (MDP). By solving this MDP, we derive the optimal transmission policy and demonstrate that it possesses a monotone structure with respect to the battery and channel states. To overcome the curse of dimensionality caused by the exponential complexity of computing the optimal policy, we propose a low-complexity algorithm, which is asymptotically optimal as the number of devices increases. Furthermore, for unknown channels and harvested energy statistics, we develop a structure-enhanced deep reinforcement learning algorithm that leverages the monotone structure of the optimal policy to improve the training performance. Finally, extensive numerical experiments on real-world datasets are presented to validate the theoretical results and corroborate the effectiveness of the proposed algorithms.