A-LAQ: Adaptive Lazily Aggregated Quantized Gradient

TL;DR

This paper introduces A-LAQ, an adaptive method for federated learning that dynamically adjusts communication bits to reduce energy consumption and improve accuracy, addressing limitations of fixed-bit approaches.

Contribution

A-LAQ extends LAQ by adaptively assigning communication bits during FL, enhancing efficiency and convergence under energy constraints.

Findings

A-LAQ reduces communication energy by up to 50%.

A-LAQ increases test accuracy by 11%.

A-LAQ demonstrates improved convergence over LAQ.

Abstract

Federated Learning (FL) plays a prominent role in solving machine learning problems with data distributed across clients. In FL, to reduce the communication overhead of data between clients and the server, each client communicates the local FL parameters instead of the local data. However, when a wireless network connects clients and the server, the communication resource limitations of the clients may prevent completing the training of the FL iterations. Therefore, communication-efficient variants of FL have been widely investigated. Lazily Aggregated Quantized Gradient (LAQ) is one of the promising communication-efficient approaches to lower resource usage in FL. However, LAQ assigns a fixed number of bits for all iterations, which may be communication-inefficient when the number of iterations is medium to high or convergence is approaching. This paper proposes Adaptive Lazily Aggregated Quantized Gradient (A-LAQ), which is a method that significantly extends LAQ by assigning an adaptive number of communication bits during the FL iterations. We train FL in an energy-constraint condition and investigate the convergence analysis for A-LAQ. The experimental results highlight that A-LAQ outperforms LAQ by up to a $50$% reduction in spent communication energy and an $11$% increase in test accuracy.