FFCL: Forward-Forward Net with Cortical Loops, Training and Inference on Edge Without Backpropagation

TL;DR

This paper introduces FFCL, an improved Forward-Forward Neural Network with cortical-like feedback loops, enabling efficient training and inference on edge devices without backpropagation, by optimizing label processing and incorporating feedback mechanisms.

Contribution

It advances the FFL method by optimizing label handling, enhancing inference, and integrating cortical-inspired feedback loops for better learning efficiency.

Findings

Improved learning performance through segregated label and feature forwarding.

Enhanced inference process with reduced computational complexity.

Feedback loops enable layers to combine complex and low-level features effectively.

Abstract

The Forward-Forward Learning (FFL) algorithm is a recently proposed solution for training neural networks without needing memory-intensive backpropagation. During training, labels accompany input data, classifying them as positive or negative inputs. Each layer learns its response to these inputs independently. In this study, we enhance the FFL with the following contributions: 1) We optimize label processing by segregating label and feature forwarding between layers, enhancing learning performance. 2) By revising label integration, we enhance the inference process, reduce computational complexity, and improve performance. 3) We introduce feedback loops akin to cortical loops in the brain, where information cycles through and returns to earlier neurons, enabling layers to combine complex features from previous layers with lower-level features, enhancing learning efficiency.