Federated Learning with Feedback Alignment

TL;DR

This paper introduces FLFA, a novel federated learning framework that uses feedback alignment to reduce local drift, improve convergence, and enhance model accuracy without extra communication or significant computational costs.

Contribution

FLFA integrates feedback alignment into federated learning to effectively mitigate local drift and improve convergence, a novel approach in FL research.

Findings

FLFA reduces local drift compared to standard FL.

FLFA demonstrates robust convergence in theoretical analysis.

Empirical results show improved accuracy with FLFA.

Abstract

Federated Learning (FL) enables collaborative training across multiple clients while preserving data privacy, yet it struggles with data heterogeneity, where clients' data are not distributed independently and identically (non-IID). This causes local drift, hindering global model convergence. To address this, we introduce Federated Learning with Feedback Alignment (FLFA), a novel framework that integrates feedback alignment into FL. FLFA uses the global model's weights as a shared feedback matrix during local training's backward pass, aligning local updates with the global model efficiently. This approach mitigates local drift with minimal additional computational cost and no extra communication overhead. Our theoretical analysis supports FLFA's design by showing how it alleviates local drift and demonstrates robust convergence for both local and global models. Empirical evaluations, including accuracy comparisons and measurements of local drift, further illustrate that FLFA can enhance other FL methods demonstrating its effectiveness.