FedPH: Privacy-enhanced Heterogeneous Federated Learning

TL;DR

FedPH introduces a privacy-preserving federated learning approach that leverages a pre-trained backbone and class embedding sharing to improve performance and privacy in heterogeneous, non-IID data environments.

Contribution

This paper proposes a novel federated learning method using class embedding sharing and differential privacy, addressing heterogeneity and privacy concerns.

Findings

Improved model performance on non-IID vehicle dataset

Effective privacy protection with minimal performance loss

Enhanced communication efficiency between server and clients

Abstract

Federated Learning is a distributed machine-learning environment that allows clients to learn collaboratively without sharing private data. This is accomplished by exchanging parameters. However, the differences in data distributions and computing resources among clients make related studies difficult. To address these heterogeneous problems, we propose a novel Federated Learning method. Our method utilizes a pre-trained model as the backbone of the local model, with fully connected layers comprising the head. The backbone extracts features for the head, and the embedding vector of classes is shared between clients to improve the head and enhance the performance of the local model. By sharing the embedding vector of classes instead of gradient-based parameters, clients can better adapt to private data, and communication between the server and clients is more effective. To protect privacy, we propose a privacy-preserving hybrid method that adds noise to the embedding vector of classes. This method has a minimal effect on the performance of the local model when differential privacy is met. We conduct a comprehensive evaluation of our approach on a self-built vehicle dataset, comparing it with other Federated Learning methods under non-independent identically distributed(Non-IID).