FedFwd: Federated Learning without Backpropagation

TL;DR

FedFwd introduces a federated learning approach that uses a backpropagation-free method, reducing computational costs and memory requirements while maintaining competitive performance on standard datasets.

Contribution

This paper presents FedFwd, a novel federated learning method that employs the Forward Forward algorithm for local training, eliminating the need for backpropagation.

Findings

Reduces computation and memory usage during training.

Achieves competitive accuracy on MNIST and CIFAR-10.

Operates effectively without backpropagation in federated settings.

Abstract

In federated learning (FL), clients with limited resources can disrupt the training efficiency. A potential solution to this problem is to leverage a new learning procedure that does not rely on backpropagation (BP). We present a novel approach to FL called FedFwd that employs a recent BP-free method by Hinton (2022), namely the Forward Forward algorithm, in the local training process. FedFwd can reduce a significant amount of computations for updating parameters by performing layer-wise local updates, and therefore, there is no need to store all intermediate activation values during training. We conduct various experiments to evaluate FedFwd on standard datasets including MNIST and CIFAR-10, and show that it works competitively to other BP-dependent FL methods.