LotteryFL: Personalized and Communication-Efficient Federated Learning with Lottery Ticket Hypothesis on Non-IID Datasets

TL;DR

LotteryFL introduces a personalized federated learning framework leveraging the Lottery Ticket hypothesis, enabling communication-efficient training on non-IID datasets with improved performance over existing methods.

Contribution

It proposes a novel approach where clients learn and communicate lottery ticket subnetworks, reducing communication costs and enhancing personalization in federated learning.

Findings

LotteryFL significantly reduces communication costs.

It outperforms existing methods in personalization on non-IID datasets.

The framework is effective on MNIST, CIFAR-10, and EMNIST datasets.

Abstract

Federated learning is a popular distributed machine learning paradigm with enhanced privacy. Its primary goal is learning a global model that offers good performance for the participants as many as possible. The technology is rapidly advancing with many unsolved challenges, among which statistical heterogeneity (i.e., non-IID) and communication efficiency are two critical ones that hinder the development of federated learning. In this work, we propose LotteryFL -- a personalized and communication-efficient federated learning framework via exploiting the Lottery Ticket hypothesis. In LotteryFL, each client learns a lottery ticket network (i.e., a subnetwork of the base model) by applying the Lottery Ticket hypothesis, and only these lottery networks will be communicated between the server and clients. Rather than learning a shared global model in classic federated learning, each client learns a personalized model via LotteryFL; the communication cost can be significantly reduced due to the compact size of lottery networks. To support the training and evaluation of our framework, we construct non-IID datasets based on MNIST, CIFAR-10 and EMNIST by taking feature distribution skew, label distribution skew and quantity skew into consideration. Experiments on these non-IID datasets demonstrate that LotteryFL significantly outperforms existing solutions in terms of personalization and communication cost.