FedLog: Personalized Federated Classification with Less Communication and More Flexibility

TL;DR

FedLog introduces a personalized federated learning approach that reduces communication costs by sharing data summaries instead of full model parameters, utilizing Bayesian inference and differential privacy for efficiency and privacy.

Contribution

It proposes a novel method that replaces model parameter sharing with data summaries encoded as sufficient statistics, enhancing communication efficiency and privacy in federated learning.

Findings

Achieves high accuracy with reduced communication overhead.

Effectively incorporates differential privacy guarantees.

Demonstrates practical efficiency in federated classification tasks.

Abstract

Federated representation learning (FRL) aims to learn personalized federated models with effective feature extraction from local data. FRL algorithms that share the majority of the model parameters face significant challenges with huge communication overhead. This overhead stems from the millions of neural network parameters and slow aggregation progress of the averaging heuristic. To reduce the overhead, we propose to share sufficient data summaries instead of raw model parameters. The data summaries encode minimal sufficient statistics of an exponential family, and Bayesian inference is utilized for global aggregation. It helps to reduce message sizes and communication frequency. To further ensure formal privacy guarantee, we extend it with differential privacy framework. Empirical results demonstrate high learning accuracy with low communication overhead of our method.