Federated Learning for Non-IID Data via Client Variance Reduction and Adaptive Server Update

TL;DR

This paper introduces ComFed, a federated learning method that addresses Non-IID data challenges by combining client-variance reduction and adaptive server updates, leading to faster convergence and improved performance.

Contribution

ComFed is a novel approach that enhances federated learning on Non-IID data through combined client variance reduction and adaptive server updates.

Findings

Improves convergence speed on Non-IID data

Outperforms state-of-the-art algorithms on CIFAR-10

Effective in accelerating federated learning processes

Abstract

Federated learning (FL) is an emerging technique used to collaboratively train a global machine learning model while keeping the data localized on the user devices. The main obstacle to FL's practical implementation is the Non-Independent and Identical (Non-IID) data distribution across users, which slows convergence and degrades performance. To tackle this fundamental issue, we propose a method (ComFed) that enhances the whole training process on both the client and server sides. The key idea of ComFed is to simultaneously utilize client-variance reduction techniques to facilitate server aggregation and global adaptive update techniques to accelerate learning. Our experiments on the Cifar-10 classification task show that ComFed can improve state-of-the-art algorithms dedicated to Non-IID data.