A Theoretical Framework for Energy-Aware Gradient Pruning in Federated Learning

TL;DR

This paper introduces a theoretical framework for energy-aware gradient pruning in federated learning, proposing a new method that considers hardware energy costs to improve efficiency.

Contribution

It formalizes energy-aware gradient pruning as an optimization problem and proposes CWMP, a novel selection rule that enhances energy efficiency in federated learning.

Findings

CWMP outperforms Top-K in energy efficiency on CIFAR-10.

CWMP is proven to be the optimal greedy solution for the energy-constrained projection.

Numerical results show improved performance-energy trade-offs.

Abstract

Federated Learning (FL) is constrained by the communication and energy limitations of decentralized edge devices. While gradient sparsification via Top-K magnitude pruning effectively reduces the communication payload, it remains inherently energy-agnostic. It assumes all parameter updates incur identical downstream transmission and memory-update costs, ignoring hardware realities. We formalize the pruning process as an energy-constrained projection problem that accounts for the hardware-level disparities between memory-intensive and compute-efficient operations during the post-backpropagation phase. We propose Cost-Weighted Magnitude Pruning (CWMP), a selection rule that prioritizes parameter updates based on their magnitude relative to their physical cost. We demonstrate that CWMP is the optimal greedy solution to this constrained projection and provide a probabilistic analysis of its global energy efficiency. Numerical results on a non-IID CIFAR-10 benchmark show that CWMP consistently establishes a superior performance-energy Pareto frontier compared to the Top-K baseline.