Adaptive Bidirectional Backpropagation: Towards Biologically Plausible Error Signal Transmission in Neural Networks

TL;DR

This paper introduces biologically plausible bidirectional learning algorithms with trainable feedback and feedforward weights, aiming to improve neural network training in a way consistent with biological neural systems.

Contribution

It proposes two novel bidirectional learning algorithms with plastic feedback weights, advancing biologically plausible error signal transmission in neural networks.

Findings

Outperforms other asymmetric BP-like methods on MNIST.

Achieves better results on CIFAR-10.

Demonstrates feasibility of biologically plausible error propagation.

Abstract

The back-propagation (BP) algorithm has been considered the de-facto method for training deep neural networks. It back-propagates errors from the output layer to the hidden layers in an exact manner using the transpose of the feedforward weights. However, it has been argued that this is not biologically plausible because back-propagating error signals with the exact incoming weights are not considered possible in biological neural systems. In this work, we propose a biologically plausible paradigm of neural architecture based on related literature in neuroscience and asymmetric BP-like methods. Specifically, we propose two bidirectional learning algorithms with trainable feedforward and feedback weights. The feedforward weights are used to relay activations from the inputs to target outputs. The feedback weights pass the error signals from the output layer to the hidden layers. Different from other asymmetric BP-like methods, the feedback weights are also plastic in our framework and are trained to approximate the forward activations. Preliminary results show that our models outperform other asymmetric BP-like methods on the MNIST and the CIFAR-10 datasets.