Physical learning beyond the quasistatic limit

TL;DR

This paper demonstrates that electrical resistor networks can learn at rates exceeding the quasistatic limit, maintaining performance up to a critical threshold, thus enabling faster physical learning systems.

Contribution

It experimentally and computationally shows that coupled learning in resistor networks can operate beyond the quasistatic limit, increasing learning speed without loss of accuracy.

Findings

Learning speeds up without increased error up to a critical threshold.

Beyond the threshold, error oscillates but learning remains successful.

Physical networks can be trained faster than their relaxation times.

Abstract

Physical networks, such as biological neural networks, can learn desired functions without a central processor, using local learning rules in space and time to learn in a fully distributed manner. Learning approaches such as equilibrium propagation, directed aging, and coupled learning similarly exploit local rules to accomplish learning in physical networks such as mechanical, flow, or electrical networks. In contrast to certain natural neural networks, however, such approaches have so far been restricted to the quasistatic limit, where they learn on time scales slow compared to their physical relaxation. This quasistatic constraint slows down learning, limiting the use of these methods as machine learning algorithms, and potentially restricting physical networks that could be used as learning platforms. Here we explore learning in an electrical resistor network that implements coupled learning, both in the lab and on the computer, at rates that range from slow to far above the quasistatic limit. We find that up to a critical threshold in the ratio of the learning rate to the physical rate of relaxation, learning speeds up without much change of behavior or error. Beyond the critical threshold, the error exhibits oscillatory dynamics but the networks still learn successfully.