Decentralised Resource Sharing in TinyML: Wireless Bilayer Gossip Parallel SGD for Collaborative Learning

TL;DR

This paper introduces GD PSGD, a hierarchical gossip-based decentralized learning framework for resource-limited edge devices, achieving comparable accuracy to centralized methods with minimal communication overhead.

Contribution

It presents a novel bilayer gossip protocol with clustering for efficient decentralized training on microcontrollers, addressing connectivity and topology challenges.

Findings

Achieves similar accuracy to centralized federated learning on IID data.

Requires only 1.8 additional rounds for convergence compared to CFL.

Maintains under 8% accuracy loss on Non-IID data with moderate imbalance.

Abstract

With the growing computational capabilities of microcontroller units (MCUs), edge devices can now support machine learning models. However, deploying decentralised federated learning (DFL) on such devices presents key challenges, including intermittent connectivity, limited communication range, and dynamic network topologies. This paper proposes a novel framework, bilayer Gossip Decentralised Parallel Stochastic Gradient Descent (GD PSGD), designed to address these issues in resource-constrained environments. The framework incorporates a hierarchical communication structure using Distributed Kmeans (DKmeans) clustering for geographic grouping and a gossip protocol for efficient model aggregation across two layers: intra-cluster and inter-cluster. We evaluate the framework's performance against the Centralised Federated Learning (CFL) baseline using the MCUNet model on the CIFAR-10 dataset under IID and Non-IID conditions. Results demonstrate that the proposed method achieves comparable accuracy to CFL on IID datasets, requiring only 1.8 additional rounds for convergence. On Non-IID datasets, the accuracy loss remains under 8\% for moderate data imbalance. These findings highlight the framework's potential to support scalable and privacy-preserving learning on edge devices with minimal performance trade-offs.