FedProf: Selective Federated Learning with Representation Profiling

TL;DR

FedProf introduces a privacy-preserving federated learning algorithm that selectively involves clients based on data quality, significantly speeding up convergence and improving accuracy by profiling data representations.

Contribution

The paper proposes a novel representation profiling and matching scheme for selective client participation in federated learning, enhancing efficiency and model quality.

Findings

Up to 2.4x faster convergence

Significant reduction in communication rounds

Improved model accuracy

Abstract

Federated Learning (FL) has shown great potential as a privacy-preserving solution to learning from decentralized data that are only accessible to end devices (i.e., clients). In many scenarios, however, a large proportion of the clients are probably in possession of low-quality data that are biased, noisy or even irrelevant. As a result, they could significantly slow down the convergence of the global model we aim to build and also compromise its quality. In light of this, we propose FedProf, a novel algorithm for optimizing FL under such circumstances without breaching data privacy. The key of our approach is a distributional representation profiling and matching scheme that uses the global model to dynamically profile data representations and allows for low-cost, lightweight representation matching. Based on the scheme we adaptively score each client and adjust its participation probability so as to mitigate the impact of low-value clients on the training process. We have conducted extensive experiments on public datasets using various FL settings. The results show that the selective behaviour of our algorithm leads to a significant reduction in the number of communication rounds and the amount of time (up to 2.4x speedup) for the global model to converge and also provides accuracy gain.