Improved Forward-Forward Contrastive Learning

TL;DR

This paper introduces an improved version of the Forward-Forward contrastive learning algorithm that eliminates the need for backpropagation, making it more biologically plausible and streamlined for neural network training.

Contribution

The authors develop a simplified, fully local update rule for Forward-Forward contrastive learning, removing the reliance on backpropagation and multi-stage training.

Findings

Achieves comparable performance without backpropagation

Simplifies the training process with local updates

Enhances biological plausibility of neural learning algorithms

Abstract

The backpropagation algorithm, or backprop, is a widely utilized optimization technique in deep learning. While there's growing evidence suggesting that models trained with backprop can accurately explain neuronal data, no backprop-like method has yet been discovered in the biological brain for learning. Moreover, employing a naive implementation of backprop in the brain has several drawbacks. In 2022, Geoffrey Hinton proposed a biologically plausible learning method known as the Forward-Forward (FF) algorithm. Shortly after this paper, a modified version called FFCL was introduced. However, FFCL had limitations, notably being a three-stage learning system where the final stage still relied on regular backpropagation. In our approach, we address these drawbacks by eliminating the last two stages of FFCL and completely removing regular backpropagation. Instead, we rely solely on local updates, offering a more biologically plausible alternative.