Training deep physical neural networks with local physical information bottleneck

TL;DR

This paper introduces the Physical Information Bottleneck (PIB), a universal, efficient framework for training deep physical neural networks across various physical platforms using information theory and local learning.

Contribution

It presents the PIB framework that enables scalable, hardware-fault-tolerant training of deep PNNs directly on physical substrates without auxiliary digital models.

Findings

PIB works with electronic memristive chips and optical platforms.

It supports supervised, unsupervised, and reinforcement learning.

PIB is robust to hardware faults and allows distributed training.

Abstract

Deep learning has revolutionized modern society but faces growing energy and latency constraints. Deep physical neural networks (PNNs) are interconnected computing systems that directly exploit analog dynamics for energy-efficient, ultrafast AI execution. Realizing this potential, however, requires universal training methods tailored to physical intricacies. Here, we present the Physical Information Bottleneck (PIB), a general and efficient framework that integrates information theory and local learning, enabling deep PNNs to learn under arbitrary physical dynamics. By allocating matrix-based information bottlenecks to each unit, we demonstrate supervised, unsupervised, and reinforcement learning across electronic memristive chips and optical computing platforms. PIB also adapts to severe hardware faults and allows for parallel training via geographically distributed resources. Bypassing auxiliary digital models and contrastive measurements, PIB recasts PNN training as an intrinsic, scalable information-theoretic process compatible with diverse physical substrates.