Personalized Online Federated Learning with Multiple Kernels

TL;DR

This paper introduces a scalable online federated multi-kernel learning algorithm that efficiently handles communication and data heterogeneity, achieving sub-linear regret and demonstrating superior performance on real datasets.

Contribution

It develops a novel scalable online federated MKL framework using random feature approximation to address communication and heterogeneity challenges.

Findings

Achieves sub-linear regret for clients with respect to their best kernels.

Outperforms existing online federated kernel learning methods on real datasets.

Effectively manages communication costs with large kernel dictionaries.

Abstract

Multi-kernel learning (MKL) exhibits well-documented performance in online non-linear function approximation. Federated learning enables a group of learners (called clients) to train an MKL model on the data distributed among clients to perform online non-linear function approximation. There are some challenges in online federated MKL that need to be addressed: i) Communication efficiency especially when a large number of kernels are considered ii) Heterogeneous data distribution among clients. The present paper develops an algorithmic framework to enable clients to communicate with the server to send their updates with affordable communication cost while clients employ a large dictionary of kernels. Utilizing random feature (RF) approximation, the present paper proposes scalable online federated MKL algorithm. We prove that using the proposed online federated MKL algorithm, each client enjoys sub-linear regret with respect to the RF approximation of its best kernel in hindsight, which indicates that the proposed algorithm can effectively deal with heterogeneity of the data distributed among clients. Experimental results on real datasets showcase the advantages of the proposed algorithm compared with other online federated kernel learning ones.