Supervised Learning Algorithms for Controlling Underactuated Dynamical Systems

TL;DR

This paper introduces two novel supervised learning algorithms capable of controlling complex underactuated dynamical systems using sparse data, with applications demonstrated across various control tasks and compared favorably to traditional methods.

Contribution

The paper presents new supervised learning algorithms that effectively control complex underactuated systems with limited data and noise, combining simplicity and versatility.

Findings

Algorithms successfully control diverse dynamical systems

Effective with sparse and noisy data

Outperform traditional control methods in tested applications

Abstract

Control of underactuated dynamical systems has been studied for decades in robotics, and is now emerging in other fields such as neuroscience. Most of the advances have been in model based control theory, which has limitations when the system under study is very complex and it is not possible to construct a model. This calls for data driven control methods like machine learning, which has spread to many fields in the recent years including control theory. However, the success of such algorithms has been dependent on availability of large datasets. Moreover, due to their black box nature, it is challenging to analyze how such algorithms work, which may be crucial in applications where failure is very costly. In this paper, we develop two related novel supervised learning algorithms. The algorithms are powerful enough to control a wide variety of complex underactuated dynamical systems, and yet have a simple and intelligent structure that allows them to work with a sparse data set even in the presence of noise. Our algorithms output a bang-bang (binary) control input by taking in feedback of the state of the dynamical system. The algorithms learn this control input by maximizing a reward function in both short and long time horizons. We demonstrate the versatility of our algorithms by applying them to a diverse range of applications including: switching between bistable states, changing the phase of an oscillator, desynchronizing a population of synchronized coupled oscillators, and stabilizing an unstable fixed point. For most of these applications we are able to reason why our algorithms work by using traditional dynamical systems and control theory. We also compare our learning algorithms with some traditional control algorithms, and reason why our algorithms work better.