A Hybrid Approach for Reinforcement Learning Using Virtual Policy Gradient for Balancing an Inverted Pendulum

TL;DR

This paper presents a hybrid reinforcement learning method using virtual policy gradients to efficiently train a neural network for balancing an inverted pendulum, with successful transfer to a physical system.

Contribution

It introduces a hybrid approach that combines simulation-based training with real-world transfer, improving training speed and robustness over existing methods.

Findings

Faster training times compared to traditional RL methods

Smoother control and better disturbance resistance

Successful transfer from simulation to physical pendulum

Abstract

Using the policy gradient algorithm, we train a single-hidden-layer neural network to balance a physically accurate simulation of a single inverted pendulum. The trained weights and biases can then be transferred to a physical agent, where they are robust enough to to balance a real inverted pendulum. This hybrid approach of training a simulation allows thousands of trial runs to be completed orders of magnitude faster than would be possible in the real world, resulting in greatly reduced training time and more iterations, producing a more robust model. When compared with existing reinforcement learning methods, the resulting control is smoother, learned faster, and able to withstand forced disturbances.