Federated Learning on Riemannian Manifolds: A Gradient-Free Projection-Based Approach

TL;DR

This paper introduces a zeroth-order, projection-based federated learning algorithm on Riemannian manifolds that efficiently handles noisy function evaluations and constrained models, with proven convergence and practical validation.

Contribution

It presents a novel zeroth-order Riemannian federated learning algorithm using a simple Euclidean perturbation, reducing computational costs and extending FL to constrained and noisy settings.

Findings

The estimator has strong approximation properties.

The algorithm achieves sublinear convergence matching first-order methods.

Validated on neural network training and attack scenarios.

Abstract

Federated learning (FL) has emerged as a powerful paradigm for collaborative model training across distributed clients while preserving data privacy. However, existing FL algorithms predominantly focus on unconstrained optimization problems with exact gradient information, limiting its applicability in scenarios where only noisy function evaluations are accessible or where model parameters are constrained. To address these challenges, we propose a novel zeroth-order projection-based algorithm on Riemannian manifolds for FL. By leveraging the projection operator, we introduce a computationally efficient zeroth-order Riemannian gradient estimator. Unlike existing estimators, ours requires only a simple Euclidean random perturbation, eliminating the need to sample random vectors in the tangent space, thus reducing computational cost. Theoretically, we first prove the approximation properties of the estimator and then establish the sublinear convergence of the proposed algorithm, matching the rate of its first-order counterpart. Numerically, we first assess the efficiency of our estimator using kernel principal component analysis. Furthermore, we apply the proposed algorithm to two real-world scenarios: zeroth-order attacks on deep neural networks and low-rank neural network training to validate the theoretical findings.