The Physical Effects of Learning

TL;DR

This paper demonstrates that learning in linear physical networks with weak inputs leaves identifiable architectural signatures in the Hessian, affecting system dimension, susceptibility, and eigenvector alignment, which can reveal training history.

Contribution

It reveals how learning imprints on the Hessian of physical systems, providing a method to detect training in physical networks.

Findings

Effective physical dimension decreases after learning

Susceptibility to perturbations increases with learning

Hessian eigenvectors align with tasks after training

Abstract

Interacting many-body physical systems ranging from neural networks in the brain to folding proteins to self-modifying electrical circuits can learn to perform diverse tasks. This learning, both in nature and in engineered systems, can occur through evolutionary selection or through dynamical rules that drive active learning from experience. Here, we show that \added{learning in linear physical networks with weak input signals} leaves architectural imprints on the Hessian of a physical system. Compared to a generic organization of the system components, (a) the effective physical dimension of the response to inputs decreases, (b) the response of physical degrees of freedom to random perturbations (or system ``susceptibility'') increases, and (c) the low-eigenvalue eigenvectors of the Hessian align with the task. Overall, these effects embody the typical scenario for learning processes in physical systems in the weak input regime, suggesting ways of discovering whether a physical network may have been trained.