Federated Learning on Riemannian Manifolds

TL;DR

This paper introduces RFedSVRG, a novel federated learning algorithm on Riemannian manifolds, addressing nonconvex constraints and demonstrating significant advantages over existing methods through theoretical analysis and experiments.

Contribution

It proposes the first federated learning algorithm specifically designed for Riemannian manifolds with convergence analysis and empirical validation.

Findings

RFedSVRG outperforms Riemannian FedAvg and FedProx in experiments.

The convergence rate of RFedSVRG is established under various scenarios.

Numerical results show significant advantages of RFedSVRG in federated PCA tasks.

Abstract

Federated learning (FL) has found many important applications in smart-phone-APP based machine learning applications. Although many algorithms have been studied for FL, to the best of our knowledge, algorithms for FL with nonconvex constraints have not been studied. This paper studies FL over Riemannian manifolds, which finds important applications such as federated PCA and federated kPCA. We propose a Riemannian federated SVRG (RFedSVRG) method to solve federated optimization over Riemannian manifolds. We analyze its convergence rate under different scenarios. Numerical experiments are conducted to compare RFedSVRG with the Riemannian counterparts of FedAvg and FedProx. We observed from the numerical experiments that the advantages of RFedSVRG are significant.