Prioritizing Gradient Sign Over Modulus: An Importance-Aware Framework for Wireless Federated Learning

TL;DR

This paper introduces SP-FL, a framework for wireless federated learning that prioritizes important gradient signs over modulus, optimizing resource allocation to enhance model accuracy under communication constraints.

Contribution

The paper proposes a novel importance-aware framework that transmits gradient signs preferentially and formulates a hierarchical resource allocation problem for wireless FL.

Findings

Up to 9.96% higher testing accuracy on CIFAR-10.

Effective resource allocation improves reliability of important gradient transmission.

Framework outperforms existing methods in resource-constrained scenarios.

Abstract

Wireless federated learning (FL) facilitates collaborative training of artificial intelligence (AI) models to support ubiquitous intelligent applications at the wireless edge. However, the inherent constraints of limited wireless resources inevitably lead to unreliable communication, which poses a significant challenge to wireless FL. To overcome this challenge, we propose Sign-Prioritized FL (SP-FL), a novel framework that improves wireless FL by prioritizing the transmission of important gradient information through uneven resource allocation. Specifically, recognizing the importance of descent direction in model updating, we transmit gradient signs in individual packets and allow their reuse for gradient descent if the remaining gradient modulus cannot be correctly recovered. To further improve the reliability of transmission of important information, we formulate a hierarchical resource allocation problem based on the importance disparity at both the packet and device levels, optimizing bandwidth allocation across multiple devices and power allocation between sign and modulus packets. To make the problem tractable, the one-step convergence behavior of SP-FL, which characterizes data importance at both levels in an explicit form, is analyzed. We then propose an alternating optimization algorithm to solve this problem using the Newton-Raphson method and successive convex approximation (SCA). Simulation results confirm the superiority of SP-FL, especially in resource-constrained scenarios, demonstrating up to 9.96\% higher testing accuracy on the CIFAR-10 dataset compared to existing methods.