Analog Physical Systems Can Exhibit Double Descent

TL;DR

This paper demonstrates that analog physical systems with self-adjusting resistive elements can exhibit the double descent phenomenon, revealing potential for energy-efficient AI hardware and insights into biological systems.

Contribution

It introduces a modified training protocol enabling analog systems to exhibit double descent, bridging physical hardware behavior with digital AI phenomena.

Findings

Analog systems can exhibit double descent with proper training.

Standard training protocols fail to produce double descent in analog systems.

Modified training protocols enable analog systems to mimic digital AI behaviors.

Abstract

An important component of the success of large AI models is double descent, in which networks avoid overfitting as they grow relative to the amount of training data, instead improving their performance on unseen data. Here we demonstrate double descent in a decentralized analog network of self-adjusting resistive elements. This system trains itself and performs tasks without a digital processor, offering potential gains in energy efficiency and speed -- but must endure component non-idealities. We find that standard training fails to yield double descent, but a modified protocol that accommodates this inherent imperfection succeeds. Our findings show that analog physical systems, if appropriately trained, can exhibit behaviors underlying the success of digital AI. Further, they suggest that biological systems might similarly benefit from over-parameterization.