Hyperspherical Forward-Forward with Prototypical Representations

TL;DR

The paper introduces Hyperspherical Forward-Forward (HFF), a reformulation of the FF algorithm that enables single-pass inference with improved speed and accuracy, including on large-scale datasets like ImageNet.

Contribution

HFF reformulates local objectives into multi-class classification in hyperspherical space, enabling faster inference and higher accuracy compared to original FF.

Findings

HFF is >40x faster than original FF.

Achieves over 25% top-1 accuracy on ImageNet-1k.

Scales effectively to modern convolutional architectures.

Abstract

The Forward-Forward (FF) algorithm presents a compelling, bio-inspired alternative to backpropagation. However, while efficient in training, it has a computationally prohibitive inference process that requires a separate forward pass for every class that is evaluated. In this work, we introduce the Hyperspherical Forward-Forward (HFF), a novel reformulation that resolves this critical bottleneck. Our core innovation is to reframe the local objective of each layer from a binary goodness-of-fit task to a direct multi-class classification problem within a hyperspherical feature space. We achieve this by learning a set of class-specific, unit-norm prototypes that act as geometric anchors and implicit negatives. This architectural innovation preserves the benefits of local training while enabling weight update and inference in a single forward pass, making it >40x faster than the original FF algorithm. Our method is simple to implement, scales effectively to modern convolutional architectures, and achieves superior accuracy on standard image classification benchmarks, closing the gap with backpropagation. Most notably, we are among the first greedy local-learning methods to report over 25% top-1 accuracy on ImageNet-1k, and 65.96% with transfer learning.