Closed-Form Feedback-Free Learning with Forward Projection

TL;DR

This paper introduces Forward Projection, a novel feedback-free learning method that trains neural networks in a single forward pass using random projections, achieving comparable or better results than traditional gradient-based methods.

Contribution

The paper proposes a new closed-form, feedback-free training approach called Forward Projection that requires only one forward pass and offers interpretability and efficiency benefits.

Findings

Achieves similar generalisation to gradient descent methods on biomedical datasets.

Requires only a single forward pass, significantly speeding up training.

Produces more generalisable models in few-shot learning scenarios.

Abstract

State-of-the-art backpropagation-free learning methods employ local error feedback to direct iterative optimisation via gradient descent. Here, we examine the more restrictive setting where retrograde communication from neuronal outputs is unavailable for pre-synaptic weight optimisation. We propose Forward Projection (FP), a randomised closed-form training method requiring only a single forward pass over the dataset without retrograde communication. FP generates target values for pre-activation membrane potentials through randomised nonlinear projections of pre-synaptic inputs and labels. Local loss functions are optimised using closed-form regression without feedback from downstream layers. A key advantage is interpretability: membrane potentials in FP-trained networks encode information interpretable layer-wise as label predictions. Across several biomedical datasets, FP achieves generalisation comparable to gradient descent-based local learning methods while requiring only a single forward propagation step, yielding significant training speedup. In few-shot learning tasks, FP produces more generalisable models than backpropagation-optimised alternatives, with local interpretation functions successfully identifying clinically salient diagnostic features.