FedLPA: One-shot Federated Learning with Layer-Wise Posterior Aggregation

TL;DR

FedLPA introduces a layer-wise posterior aggregation technique for one-shot federated learning, effectively handling data heterogeneity without extra data or privacy risks, leading to improved global model accuracy.

Contribution

The paper proposes FedLPA, a novel one-shot federated learning method that uses layer-wise Laplace approximation for posterior aggregation, addressing non-IID data challenges.

Findings

FedLPA outperforms existing methods in accuracy across multiple benchmarks.

It effectively handles high data heterogeneity without additional data sharing.

The approach maintains privacy by not exposing label distributions.

Abstract

Efficiently aggregating trained neural networks from local clients into a global model on a server is a widely researched topic in federated learning. Recently, motivated by diminishing privacy concerns, mitigating potential attacks, and reducing communication overhead, one-shot federated learning (i.e., limiting client-server communication into a single round) has gained popularity among researchers. However, the one-shot aggregation performances are sensitively affected by the non-identical training data distribution, which exhibits high statistical heterogeneity in some real-world scenarios. To address this issue, we propose a novel one-shot aggregation method with layer-wise posterior aggregation, named FedLPA. FedLPA aggregates local models to obtain a more accurate global model without requiring extra auxiliary datasets or exposing any private label information, e.g., label distributions. To effectively capture the statistics maintained in the biased local datasets in the practical non-IID scenario, we efficiently infer the posteriors of each layer in each local model using layer-wise Laplace approximation and aggregate them to train the global parameters. Extensive experimental results demonstrate that FedLPA significantly improves learning performance over state-of-the-art methods across several metrics.