Over-the-Air Federated Learning via Second-Order Optimization

TL;DR

This paper introduces a novel over-the-air second-order federated learning algorithm that reduces communication rounds and latency by leveraging wireless channel properties, with proven convergence and system optimization.

Contribution

It proposes the first over-the-air second-order federated optimization method, improving communication efficiency and convergence speed over existing first-order approaches.

Findings

Achieves linear-quadratic convergence rate.

Reduces communication rounds compared to first-order methods.

Demonstrates system efficiency through numerical results.

Abstract

Federated learning (FL) is a promising learning paradigm that can tackle the increasingly prominent isolated data islands problem while keeping users' data locally with privacy and security guarantees. However, FL could result in task-oriented data traffic flows over wireless networks with limited radio resources. To design communication-efficient FL, most of the existing studies employ the first-order federated optimization approach that has a slow convergence rate. This however results in excessive communication rounds for local model updates between the edge devices and edge server. To address this issue, in this paper, we instead propose a novel over-the-air second-order federated optimization algorithm to simultaneously reduce the communication rounds and enable low-latency global model aggregation. This is achieved by exploiting the waveform superposition property of a multi-access channel to implement the distributed second-order optimization algorithm over wireless networks. The convergence behavior of the proposed algorithm is further characterized, which reveals a linear-quadratic convergence rate with an accumulative error term in each iteration. We thus propose a system optimization approach to minimize the accumulated error gap by joint device selection and beamforming design. Numerical results demonstrate the system and communication efficiency compared with the state-of-the-art approaches.