Constrained Attractor Selection Using Deep Reinforcement Learning

TL;DR

This paper explores using deep reinforcement learning methods, specifically CEM and DDPG, to control multi-stability in nonlinear systems with constraints, demonstrated on a Duffing oscillator.

Contribution

It introduces a framework applying CEM and DDPG for attractor selection in constrained nonlinear systems, highlighting DDPG's advantages.

Findings

Both methods successfully achieve attractor selection.

DDPG offers faster learning and smoother control.

Methods perform similarly in success rate.

Abstract

This paper describes an approach for attractor selection (or multi-stability control) in nonlinear dynamical systems with constrained actuation. Attractor selection is obtained using two different deep reinforcement learning methods: 1) the cross-entropy method (CEM) and 2) the deep deterministic policy gradient (DDPG) method. The framework and algorithms for applying these control methods are presented. Experiments were performed on a Duffing oscillator, as it is a classic nonlinear dynamical system with multiple attractors. Both methods achieve attractor selection under various control constraints. While these methods have nearly identical success rates, the DDPG method has the advantages of a high learning rate, low performance variance, and a smooth control approach. This study demonstrates the ability of two reinforcement learning approaches to achieve constrained attractor selection.