Mixed-Precision Federated Learning via Multi-Precision Over-The-Air Aggregation

TL;DR

This paper introduces a mixed-precision federated learning framework over-the-air that accommodates clients with varying hardware capabilities, improving efficiency and accuracy in heterogeneous edge environments.

Contribution

It proposes a novel multi-precision gradient modulation scheme enabling mixed-precision OTA-FL, addressing compatibility and efficiency challenges in heterogeneous client settings.

Findings

Over 10% performance gain with 4-bit clients compared to standard precision.

Over 65% energy savings with ultra low precision clients.

Effective compatibility of mixed-precision updates with OTA aggregation.

Abstract

Over-the-Air Federated Learning (OTA-FL) is a privacy-preserving distributed learning mechanism, by aggregating updates in the electromagnetic channel rather than at the server. A critical research gap in existing OTA-FL research is the assumption of homogeneous client computational bit precision. While in real world application, clients with varying hardware resources may exploit approximate computing (AxC) to operate at different bit precisions optimized for energy and computational efficiency. And model updates of various precisions amongst clients poses an open challenge for OTA-FL, as it is incompatible in the wireless modulation superposition. Here, we propose an mixed-precision OTA-FL framework of clients with multiple bit precisions, demonstrating the following innovations: (i) the superior trade-off for both server and clients within the constraints of varying edge computing capabilities, energy efficiency, and learning accuracy requirements comparing to homogeneous client bit precision, and (ii) a multi-precision gradient modulation scheme to ensure compatibility with OTA aggregation and eliminate the overheads of precision conversion. Through case study with real world data, we validate our modulation scheme that enables AxC based mixed-precision OTA-FL. In comparison to homogeneous standard precision of 32-bit and 16-bit, our framework presents more than 10% in 4-bit ultra low precision client performance and over 65%and 13% of energy savings respectively. This demonstrates the great potential of our mixed-precision OTA-FL approach in heterogeneous edge computing environments.