Addressing Heterogeneity in Federated Learning via Distributional Transformation

TL;DR

This paper introduces DisTrans, a novel framework that enhances federated learning by applying distributional transformations to address data heterogeneity across clients, leading to improved model accuracy.

Contribution

DisTrans is a new framework that optimizes distributional offsets for each client and aggregates them to better handle heterogeneity in federated learning.

Findings

DisTrans outperforms existing FL methods on benchmark datasets.

It effectively handles varying degrees of data heterogeneity.

The approach improves model accuracy in heterogeneous settings.

Abstract

Federated learning (FL) allows multiple clients to collaboratively train a deep learning model. One major challenge of FL is when data distribution is heterogeneous, i.e., differs from one client to another. Existing personalized FL algorithms are only applicable to narrow cases, e.g., one or two data classes per client, and therefore they do not satisfactorily address FL under varying levels of data heterogeneity. In this paper, we propose a novel framework, called DisTrans, to improve FL performance (i.e., model accuracy) via train and test-time distributional transformations along with a double-input-channel model structure. DisTrans works by optimizing distributional offsets and models for each FL client to shift their data distribution, and aggregates these offsets at the FL server to further improve performance in case of distributional heterogeneity. Our evaluation on multiple benchmark datasets shows that DisTrans outperforms state-of-the-art FL methods and data augmentation methods under various settings and different degrees of client distributional heterogeneity.