Experimental Study on Reinforcement Learning-based Control of an Acrobot

TL;DR

This paper investigates how reinforcement learning can be used to control an Acrobot in embedded systems, focusing on energy and angular velocity regulation through extensive simulations and experiments.

Contribution

It provides novel insights into RL control of an Acrobot, including effects of state discretization, episode length, and parameter variations, with practical experimental validation.

Findings

RL successfully controls Acrobot energy and velocity

Parameter variations significantly affect RL performance

Experimental results validate simulation insights

Abstract

We present computational and experimental results on how artificial intelligence (AI) learns to control an Acrobot using reinforcement learning (RL). Thereby the experimental setup is designed as an embedded system, which is of interest for robotics and energy harvesting applications. Specifically, we study the control of angular velocity of the Acrobot, as well as control of its total energy, which is the sum of the kinetic and the potential energy. By this means the RL algorithm is designed to drive the angular velocity or the energy of the first pendulum of the Acrobot towards a desired value. With this, libration or full rotation of the unactuated pendulum of the Acrobot is achieved. Moreover, investigations of the Acrobot control are carried out, which lead to insights about the influence of the state space discretization, the episode length, the action space or the mass of the driven pendulum on the RL control. By further numerous simulations and experiments the effects of parameter variations are evaluated.