Dynamically writing coupled memories using a reinforcement learning agent, meeting physical bounds

TL;DR

This paper demonstrates how a reinforcement learning agent can control a multi-bit mechanical system to maximize memory capacity by exploiting dynamical responses, surpassing traditional quasi-static methods and approaching physical limits.

Contribution

It introduces a model of coupled bi-stable springs and shows how reinforcement learning can optimize memory writing, including system design and control time, using transfer learning.

Findings

RL agent restores full memory capacity in a multi-bit system.

Transfer learning accelerates training and improves convergence.

Control time depends non-monotonically on dissipation, matching theoretical scaling.

Abstract

Traditional memory writing operations proceed one bit at a time, where e.g. an individual magnetic domain is force-flipped by a localized external field. One way to increase material storage capacity would be to write several bits at a time in the bulk of the material. However, the manipulation of bits is commonly done through quasi-static operations. While simple to model, this method is known to reduce memory capacity. In this paper, we demonstrate how a reinforcement learning agent can exploit the dynamical response of a simple multi-bit mechanical system to restore its memory to full capacity. To do so, we introduce a model framework consisting of a chain of bi-stable springs, which is manipulated on one end by the external action of the agent. We show that the agent manages to learn how to reach all available states for three springs, even though some states are not reachable through adiabatic manipulation, and that both the training speed and convergence within physical parameter space are improved using transfer learning techniques. Interestingly, the agent also points to an optimal design of the system in terms of writing time. In fact, it appears to learn how to take advantage of the underlying physics: the control time exhibits a non-monotonic dependence on the internal dissipation, reaching a minimum at a cross-over shown to verify a mechanically motivated scaling relation.