Random feedback weights support learning in deep neural networks

TL;DR

This paper introduces a simple, biologically plausible learning algorithm using random feedback weights, enabling deep neural networks to learn effectively without exact weight transport, challenging traditional backpropagation assumptions.

Contribution

It presents a novel learning method using random feedback weights that performs comparably to backpropagation, offering insights into plausible neural learning mechanisms.

Findings

Random feedback weights support effective learning in deep networks.

The method achieves accuracy comparable to backpropagation.

Provides a plausible neural mechanism for error-driven learning.

Abstract

The brain processes information through many layers of neurons. This deep architecture is representationally powerful, but it complicates learning by making it hard to identify the responsible neurons when a mistake is made. In machine learning, the backpropagation algorithm assigns blame to a neuron by computing exactly how it contributed to an error. To do this, it multiplies error signals by matrices consisting of all the synaptic weights on the neuron's axon and farther downstream. This operation requires a precisely choreographed transport of synaptic weight information, which is thought to be impossible in the brain. Here we present a surprisingly simple algorithm for deep learning, which assigns blame by multiplying error signals by random synaptic weights. We show that a network can learn to extract useful information from signals sent through these random feedback connections. In essence, the network learns to learn. We demonstrate that this new mechanism performs as quickly and accurately as backpropagation on a variety of problems and describe the principles which underlie its function. Our demonstration provides a plausible basis for how a neuron can be adapted using error signals generated at distal locations in the brain, and thus dispels long-held assumptions about the algorithmic constraints on learning in neural circuits.