Biological Plausibility and Representational Alignment of Feedback Alignment in Convolutional Networks

TL;DR

This study compares feedback alignment and backpropagation in convolutional networks, showing modified FA can produce similar internal representations to BP, with implications for biological plausibility.

Contribution

It demonstrates that modified feedback alignment algorithms can achieve representational similarity to backpropagation in convolutional networks.

Findings

Modified FA converges on representations similar to BP.

Representational geometry of FA mimics that of BP.

Modified FA maintains biological plausibility while achieving effective learning.

Abstract

The feedback alignment (FA) algorithm offers a biologically plausible alternative to backpropagation (BP) for training neural networks yet notably fails to scale to convolutional architectures. Modifications have been proposed to address this limitation, but at questionable cost to biological plausibility. In this paper, we evaluate five learning algorithms including modified FA and standard BP, applied to the same convolutional architecture with the CIFAR-10 dataset. We provide a tripartite comparative analysis focusing on biological plausibility, interpretability, and computational complexity. Our results indicate that modified FA algorithms converge on internal representations that are structurally similar to those produced by backpropagation. In particular, it appears the functional success of modified FA algorithms may be rooted in their ability to mimic the representational geometry of backpropagation, converging on similar representations despite relying on fundamentally different weight update mechanisms.