Physical learning of power-efficient solutions

TL;DR

This paper explores how physical learning machines, specifically neuromorphic hardware, can reduce energy consumption in machine learning by optimizing initial conditions and learning algorithms, balancing accuracy and power efficiency.

Contribution

It introduces a new learning algorithm and analytical framework for physical learning machines that optimize energy use while managing accuracy trade-offs.

Findings

Good initial conditions improve energy efficiency.

Trade-off exists between power reduction and solution accuracy.

Practical procedure to balance error and power consumption.

Abstract

As the size and ubiquity of artificial intelligence and computational machine learning (ML) models grow, their energy consumption for training and use is rapidly becoming economically and environmentally unsustainable. Neuromorphic computing, or the implementation of ML in hardware, has the potential to reduce this cost. In particular, recent laboratory prototypes of self-learning electronic circuits, examples of ``physical learning machines," open the door to analog hardware that directly employs physics to learn desired functions from examples. In this work, we show that this hardware platform allows for even further reduction of energy consumption by using good initial conditions as well as a new learning algorithm. Using analytical calculations, simulation and experiment, we show that a trade-off emerges when learning dynamics attempt to minimize both the error and the power consumption of the solution--greater power reductions can be achieved at the cost of decreasing solution accuracy. Finally, we demonstrate a practical procedure to weigh the relative importance of error and power minimization, improving power efficiency given a specific tolerance to error.