The Integrated Forward-Forward Algorithm: Integrating Forward-Forward and Shallow Backpropagation With Local Losses

TL;DR

This paper introduces an integrated training algorithm combining Forward-Forward and shallow backpropagation, aiming for more biologically plausible neural networks with improved robustness and applicability to various structures.

Contribution

It proposes a novel integrated training method that merges FFA and shallow backpropagation, enhancing biological plausibility and network robustness.

Findings

Outperforms FFA on MNIST classification

Demonstrates increased noise resilience compared to backpropagation

Applicable to diverse neural network architectures

Abstract

The backpropagation algorithm, despite its widespread use in neural network learning, may not accurately emulate the human cortex's learning process. Alternative strategies, such as the Forward-Forward Algorithm (FFA), offer a closer match to the human cortex's learning characteristics. However, the original FFA paper and related works on the Forward-Forward Algorithm only mentioned very limited types of neural network mechanisms and may limit its application and effectiveness. In response to these challenges, we propose an integrated method that combines the strengths of both FFA and shallow backpropagation, yielding a biologically plausible neural network training algorithm which can also be applied to various network structures. We applied this integrated approach to the classification of the Modified National Institute of Standards and Technology (MNIST) database, where it outperformed FFA and demonstrated superior resilience to noise compared to backpropagation. We show that training neural networks with the Integrated Forward-Forward Algorithm has the potential of generating neural networks with advantageous features like robustness.