Multi-Task Model Personalization for Federated Supervised SVM in Heterogeneous Networks

TL;DR

This paper introduces an efficient federated learning method for personalized SVMs that handles heterogeneous data and devices, improving convergence and privacy in distributed environments.

Contribution

It proposes a novel ADMM-based distributed algorithm for federated SVMs with privacy-preserving random masking, addressing heterogeneity and convergence issues.

Findings

Enhanced convergence speed in heterogeneous settings

Effective privacy preservation through random masking

Robust performance despite device and data variability

Abstract

Federated systems enable collaborative training on highly heterogeneous data through model personalization, which can be facilitated by employing multi-task learning algorithms. However, significant variation in device computing capabilities may result in substantial degradation in the convergence rate of training. To accelerate the learning procedure for diverse participants in a multi-task federated setting, more efficient and robust methods need to be developed. In this paper, we design an efficient iterative distributed method based on the alternating direction method of multipliers (ADMM) for support vector machines (SVMs), which tackles federated classification and regression. The proposed method utilizes efficient computations and model exchange in a network of heterogeneous nodes and allows personalization of the learning model in the presence of non-i.i.d. data. To further enhance privacy, we introduce a random mask procedure that helps avoid data inversion. Finally, we analyze the impact of the proposed privacy mechanisms and participant hardware and data heterogeneity on the system performance.