Client Selection for Federated Bayesian Learning

TL;DR

This paper introduces client selection schemes for federated Bayesian learning using DSVGD, improving convergence and communication efficiency by selecting the most informative clients based on KSD and HIP metrics.

Contribution

The paper proposes two novel client selection schemes for DSVGD in federated learning, with theoretical bounds and empirical validation showing improved convergence and efficiency.

Findings

KSD-based scheme accelerates convergence.

HIP-based scheme enhances stability.

Both schemes outperform conventional methods.

Abstract

Distributed Stein Variational Gradient Descent (DSVGD) is a non-parametric distributed learning framework for federated Bayesian learning, where multiple clients jointly train a machine learning model by communicating a number of non-random and interacting particles with the server. Since communication resources are limited, selecting the clients with most informative local learning updates can improve the model convergence and communication efficiency. In this paper, we propose two selection schemes for DSVGD based on Kernelized Stein Discrepancy (KSD) and Hilbert Inner Product (HIP). We derive the upper bound on the decrease of the global free energy per iteration for both schemes, which is then minimized to speed up the model convergence. We evaluate and compare our schemes with conventional schemes in terms of model accuracy, convergence speed, and stability using various learning tasks and datasets.