Adaptive Control of an Inverted Pendulum by a Reinforcement Learning-based LQR Method

TL;DR

This paper introduces an adaptive control method for inverted pendulums using reinforcement learning combined with LQR, achieving fast stabilization without needing a detailed model or extensive tuning, and adapting to parameter changes online.

Contribution

The paper presents a novel LQR-based reinforcement learning approach that improves stability speed and adaptability for inverted pendulum control without requiring a mathematical model.

Findings

Fast stabilization achieved in numerical experiments

No need for a mathematical model or extensive hyperparameter tuning

System adapts online to parametric changes

Abstract

Inverted pendulums constitute one of the popular systems for benchmarking control algorithms. Several methods have been proposed for the control of this system, the majority of which rely on the availability of a mathematical model. However, deriving a mathematical model using physical parameters or system identification techniques requires manual effort. Moreover, the designed controllers may perform poorly if system parameters change. To mitigate these problems, recently, some studies used Reinforcement Learning (RL) based approaches for the control of inverted pendulum systems. Unfortunately, these methods suffer from slow convergence and local minimum problems. Moreover, they may require hyperparameter tuning which complicates the design process significantly. To alleviate these problems, the present study proposes an LQR-based RL method for adaptive balancing control of an inverted pendulum. As shown by numerical experiments, the algorithm stabilizes the system very fast without requiring a mathematical model or extensive hyperparameter tuning. In addition, it can adapt to parametric changes online.