Mono-Forward: Revisiting Forward-Forward through Objective-Locality Decomposition

TL;DR

This paper introduces Mono-Forward, a local-learning algorithm that improves upon the Forward-Forward method by replacing its goodness objective with a local cross-entropy, achieving competitive accuracy with less memory.

Contribution

It revisits the Forward-Forward algorithm, identifies the impact of the goodness objective, and proposes Mono-Forward, a simplified, more effective local learning method.

Findings

Mono-Forward outperforms vanilla FF across models.

Mono-Forward achieves better accuracy than FF variants.

On PathMNIST, Mono-Forward surpasses backpropagation with less memory.

Abstract

Backpropagation remains the dominant algorithm for training deep neural networks, but it incurs substantial memory overhead and relies on global error propagation, which is often regarded as biologically implausible. The Forward-Forward (FF) algorithm is an appealing local-learning alternative to backpropagation, yet it still lags behind backpropagation in accuracy. A central unresolved question is whether this gap arises from FF's locality or from the positive-negative double-pass goodness objective used to train each layer. In this work, we revisit FF under the supervised setting through a decomposition that separates these two design choices. Our analysis suggests that FF's performance limitations are not explained by locality alone, but are also likely influenced by its goodness objective. Motivated by this view, we introduce Mono-Forward (MF), a simplification of FF that preserves its locality while replacing the contrastive goodness objective with a standard multi-class cross-entropy objective applied locally at each layer, serving as a controlled baseline for evaluating local learning under a standard classification objective. Across MLPs and convolutional networks, MF outperforms vanilla FF and remains competitive in multiple FF variants. On MLP-Mixers, MF achieves stronger results on PathMNIST than backpropagation while requiring only 31% of backpropagation's memory.