Constrained Over-the-Air Model Updating for Wireless Online Federated Learning with Delayed Information

TL;DR

This paper introduces COMUDO, a novel method for wireless online federated learning that effectively manages delayed information and power constraints, resulting in improved model accuracy over existing approaches.

Contribution

The paper proposes COMUDO, a new over-the-air model updating algorithm with virtual queues and Lyapunov analysis to handle delays and constraints in wireless federated learning.

Findings

COMUDO achieves lower regret bounds compared to existing methods.

Simulation shows COMUDO improves accuracy, especially under low-power conditions.

The method effectively controls constraint violations in practical wireless settings.

Abstract

We study online federated learning over a wireless network, where the central server updates an online global model sequence to minimize the time-varying loss of multiple local devices over time. The server updates the global model through over-the-air model-difference aggregation from the local devices over a noisy multiple-access fading channel. We consider the practical scenario where information on both the local loss functions and the channel states is delayed, and each local device is under a time-varying power constraint. We propose Constrained Over-the-air Model Updating with Delayed infOrmation (COMUDO), where a new lower-and-upper-bounded virtual queue is introduced to counter the delayed information and control the hard constraint violation. We show that its local model updates can be efficiently computed in closed-form expressions. Furthermore, through a new Lyapunov drift analysis, we show that COMUDO provides bounds on the dynamic regret, static regret, and hard constraint violation. Simulation results on image classification tasks under practical wireless network settings show substantial accuracy gain of COMUDO over state-of-the-art approaches, especially in the low-power region.