Knowledge-aided Federated Learning for Energy-limited Wireless Networks

TL;DR

This paper introduces a knowledge-aided federated learning framework that reduces communication costs and supports heterogeneous models in energy-limited wireless networks, with proven convergence and improved performance.

Contribution

It proposes a novel KFL framework that aggregates high-level features instead of models, enabling independent model design and reducing communication overhead.

Findings

Reduces over 99% communication overhead compared to traditional FL.

Achieves better learning performance on MNIST and CIFAR-10 datasets.

Provides theoretical convergence analysis and optimal resource allocation strategies.

Abstract

The conventional model aggregation-based federated learning (FL) approach requires all local models to have the same architecture, which fails to support practical scenarios with heterogeneous local models. Moreover, frequent model exchange is costly for resource-limited wireless networks since modern deep neural networks usually have over a million parameters. To tackle these challenges, we propose a novel knowledge-aided FL (KFL) framework, which aggregates light high-level data features, namely knowledge, in the per-round learning process. This framework allows devices to design their machine-learning models independently and reduces the communication overhead in the training process. We then theoretically analyze the convergence bound of the proposed framework, revealing that scheduling more data volume in each round helps to improve the learning performance. In addition, large data volume should be scheduled in early rounds if the total scheduled data volume during the entire learning course is fixed. Inspired by this, we define a new objective function, i.e., the weighted scheduled data sample volume, to transform the inexplicit global loss minimization problem into a tractable one for device scheduling, bandwidth allocation, and power control. To deal with unknown time-varying wireless channels, we transform the considered problem into a deterministic problem for each round with the assistance of the Lyapunov optimization framework. Then, we derive the optimal bandwidth allocation and power control solution and develop an efficient online device scheduling algorithm to achieve an energy-learning trade-off in the learning process. Experimental results on MNIST and CIFAR-10 show that the proposed KFL is capable of reducing over 99% communication overhead while achieving better learning performance than the conventional model aggregation-based algorithms.