Riemannian Federated Learning via Averaging Gradient Streams

TL;DR

This paper introduces RFedAGS, a Riemannian federated learning algorithm that effectively handles partial participation and data heterogeneity, with proven convergence properties and strong empirical performance.

Contribution

The paper proposes RFedAGS, a novel Riemannian federated learning algorithm utilizing gradient stream averaging, addressing partial participation and heterogeneity challenges.

Findings

Proven global convergence and sublinear rates under decaying step sizes.

Convergence to a neighborhood of stationary points with fixed step sizes.

Demonstrated strong empirical performance on synthetic and real data.

Abstract

Federated learning (FL) as a distributed learning paradigm has a significant advantage in addressing large-scale machine learning tasks. In the Euclidean setting, FL algorithms have been extensively studied with both theoretical and empirical success. However, there exist few works that investigate federated learning algorithms in the Riemannian setting. In particular, critical challenges such as partial participation and data heterogeneity among agents are not explored in the Riemannian federated setting. This paper presents and analyzes a Riemannian FL algorithm, called RFedAGS, based on a new efficient server aggregation -- averaging gradient streams, which can simultaneously handle partial participation and data heterogeneity. We theoretically show that the proposed RFedAGS has global convergence and sublinear convergence rate under decaying step sizes cases; and converges sublinearly/linearly to a neighborhood of a stationary point/solution under fixed step sizes cases. These analyses are based on a vital and non-trivial assumption induced by partial participation, which is shown to hold with high probability. Extensive experiments conducted on synthetic and real-world data demonstrate the good performance of RFedAGS.