Peer-to-Peer Deep Learning for Beyond-5G IoT

TL;DR

This paper introduces P2PL, a decentralized peer-to-peer deep learning method for beyond-5G IoT environments that eliminates the need for central coordination, enabling scalable, private, and robust distributed training.

Contribution

P2PL is a novel decentralized deep learning algorithm that uses max norm synchronization and local peer communication, avoiding reliance on edge servers or cloud coordination.

Findings

Achieves the same test performance as federated and centralized training.

Effective in diverse network topologies and with intermittent communication.

Handles non-IID data distributions effectively.

Abstract

We present P2PL, a practical multi-device peer-to-peer deep learning algorithm that, unlike the federated learning paradigm, does not require coordination from edge servers or the cloud. This makes P2PL well-suited for the sheer scale of beyond-5G computing environments like smart cities that otherwise create range, latency, bandwidth, and single point of failure issues for federated approaches. P2PL introduces max norm synchronization to catalyze training, retains on-device deep model training to preserve privacy, and leverages local inter-device communication to implement distributed consensus. Each device iteratively alternates between two phases: 1) on-device learning and 2) peer-to-peer cooperation where they combine model parameters with nearby devices. We empirically show that all participating devices achieve the same test performance attained by federated and centralized training -- even with 100 devices and relaxed singly stochastic consensus weights. We extend these experimental results to settings with diverse network topologies, sparse and intermittent communication, and non-IID data distributions.