Distributed Learning for Time-varying Networks: A Scalable Design

TL;DR

This paper introduces a scalable distributed learning framework tailored for time-varying wireless networks, leveraging deep neural network structures to enhance convergence and performance despite network topology changes.

Contribution

It proposes a novel DNN-based distributed learning framework that accounts for wireless network dynamics using permutation invariance, improving scalability and convergence.

Findings

Framework outperforms baseline methods in simulations

Scalable DNN design adapts to network topology changes

Model aggregation improves learning convergence

Abstract

The wireless network is undergoing a trend from "onnection of things" to "connection of intelligence". With data spread over the communication networks and computing capability enhanced on the devices, distributed learning becomes a hot topic in both industrial and academic communities. Many frameworks, such as federated learning and federated distillation, have been proposed. However, few of them takes good care of obstacles such as the time-varying topology resulted by the characteristics of wireless networks. In this paper, we propose a distributed learning framework based on a scalable deep neural network (DNN) design. By exploiting the permutation equivalence and invariance properties of the learning tasks, the DNNs with different scales for different clients can be built up based on two basic parameter sub-matrices. Further, model aggregation can also be conducted based on these two sub-matrices to improve the learning convergence and performance. Finally, simulation results verify the benefits of the proposed framework by compared with some baselines.