Non-equilibrium physics: from spin glasses to machine and neural learning

TL;DR

This paper explores how disordered many-body systems exhibit emergent intelligent behaviors, linking physical dynamics with learning mechanisms to inform the design of novel AI computational substrates.

Contribution

It provides a framework connecting physical dynamics and learning mechanisms in disordered systems, advancing understanding of emergent intelligence beyond neural networks.

Findings

Relationships between learning mechanisms and physical dynamics identified

Guiding principles for designing intelligent physical systems proposed

Broader understanding of computational substrates for AI developed

Abstract

Disordered many-body systems exhibit a wide range of emergent phenomena across different scales. These complex behaviors can be utilized for various information processing tasks such as error correction, learning, and optimization. Despite the empirical success of utilizing these systems for intelligent tasks, the underlying principles that govern their emergent intelligent behaviors remain largely unknown. In this thesis, we aim to characterize such emergent intelligence in disordered systems through statistical physics. We chart a roadmap for our efforts in this thesis based on two axes: learning mechanisms (long-term memory vs. working memory) and learning dynamics (artificial vs. natural). Throughout our journey, we uncover relationships between learning mechanisms and physical dynamics that could serve as guiding principles for designing intelligent systems. We hope that our investigation into the emergent intelligence of seemingly disparate learning systems can expand our current understanding of intelligence beyond neural systems and uncover a wider range of computational substrates suitable for AI applications.