Learning to control non-equilibrium dynamics using local imperfect gradients

TL;DR

This paper introduces a biologically plausible, local iterative method for controlling non-equilibrium dynamical systems using imperfect feedback, supported by theoretical analysis and numerical experiments.

Contribution

It proposes a novel local, iterative control protocol for non-equilibrium systems that does not rely on full backpropagation or model-based representations.

Findings

The method converges locally for conservative force fields.

It effectively controls active nematic systems.

The approach is biologically plausible and information-efficient.

Abstract

Standard approaches to controlling dynamical systems involve biologically implausible steps such as backpropagation of errors or intermediate model-based system representations. Recent advances in machine learning have shown that "imperfect" feedback of errors during training can yield test performance that is similar to using full backpropagated errors, provided that the two error signals are at least somewhat aligned. Inspired by such methods, we introduce an iterative, spatiotemporally local protocol to learn driving forces and control non-equilibrium dynamical systems using imperfect feedback signals. We present numerical experiments and theoretical justification for several examples. For systems in conservative force fields that are driven by external time-dependent protocols, our update rules resemble a dynamical version of contrastive divergence. We appeal to linear response theory to establish that our imperfect update rules are locally convergent for these conservative systems. Finally, we show that similar local update rules can also solve dynamical control problems for non-conservative systems, and we illustrate this in the non-trivial example of active nematics. Our updates allow learning spatiotemporal activity fields that pull topological defects along desired trajectories in the active nematic fluid. These imperfect feedback methods are information efficient and in principle biologically plausible, and they can help extend recent methods of decentralized training for physical materials into dynamical settings.