Trainable and Dynamic Computing: Error Backpropagation through Physical Media

TL;DR

This paper proposes a novel physical system for neural network training that performs error backpropagation directly through physical media, potentially enabling scalable and faster analog computing for machine learning.

Contribution

It introduces a physical linear dynamic system with nonlinear feedback capable of performing both neural computation and error backpropagation physically, advancing analog neural network training.

Findings

Physical backpropagation speeds up training process

Demonstrated with experimental and conceptual examples

Potential for scalable, fully dynamic analog computers

Abstract

Machine learning algorithms, and more in particular neural networks, arguably experience a revolution in terms of performance. Currently, the best systems we have for speech recognition, computer vision and similar problems are based on neural networks, trained using the half-century old backpropagation algorithm. Despite the fact that neural networks are a form of analog computers, they are still implemented digitally for reasons of convenience and availability. In this paper we demonstrate how we can design physical linear dynamic systems with non-linear feedback as a generic platform for dynamic, neuro-inspired analog computing. We show that a crucial advantage of this setup is that the error backpropagation can be performed physically as well, which greatly speeds up the optimisation process. As we show in this paper, using one experimentally validated and one conceptual example, such systems may be the key to providing a relatively straightforward mechanism for constructing highly scalable, fully dynamic analog computers.