Energy-Efficient Deep Learning Without Backpropagation: A Rigorous Evaluation of Forward-Only Algorithms

TL;DR

This paper demonstrates that the Mono-Forward algorithm, a backpropagation-free method, outperforms traditional backpropagation in accuracy and efficiency for MLPs, challenging long-held assumptions in deep learning.

Contribution

It provides the first rigorous, hardware-validated evidence that a backpropagation-free algorithm can surpass BP in accuracy and efficiency on standard neural network architectures.

Findings

MF outperforms optimized BP in accuracy on MLPs

MF reduces energy consumption by up to 41%

MF accelerates training by up to 34%

Abstract

The long-held assumption that backpropagation (BP) is essential for state-of-the-art performance is challenged by this work. We present rigorous, hardware-validated evidence that the Mono-Forward (MF) algorithm, a backpropagation-free method, consistently surpasses an optimally tuned BP baseline in classification accuracy on its native Multi-Layer Perceptron (MLP) architectures. This superior generalization is achieved with profound efficiency gains, including up to 41% less energy consumption and up to 34% faster training. Our analysis, which charts an evolutionary path from Geoffrey Hinton's Forward-Forward (FF) to the Cascaded Forward (CaFo) and finally to MF, is grounded in a fair comparative framework using identical architectures and universal hyperparameter optimization. We further provide a critical re-evaluation of memory efficiency in BP-free methods, empirically demonstrating that practical overhead can offset theoretical gains. Ultimately, this work establishes MF as a practical, high-performance, and sustainable alternative to BP for MLPs.