Learning Ball-balancing Robot Through Deep Reinforcement Learning

TL;DR

This paper introduces a compound control approach combining traditional feedback control with deep reinforcement learning to enhance the balancing capabilities of a ball-balancing robot, enabling it to recover from larger tilts.

Contribution

The paper proposes a novel compound controller that integrates RL with conventional control, improving the recovery performance of the ballbot under large initial tilts.

Findings

The compound controller enables the ballbot to recover from larger initial tilts.

Simulation results show improved balance stability with the proposed method.

The approach effectively handles contacts and collisions during recovery.

Abstract

The ball-balancing robot (ballbot) is a good platform to test the effectiveness of a balancing controller. Considering balancing control, conventional model-based feedback control methods have been widely used. However, contacts and collisions are difficult to model, and often lead to failure in balancing control, especially when the ballbot tilts a large angle. To explore the maximum initial tilting angle of the ballbot, the balancing control is interpreted as a recovery task using Reinforcement Learning (RL). RL is a powerful technique for systems that are difficult to model, because it allows an agent to learn policy by interacting with the environment. In this paper, by combining the conventional feedback controller with the RL method, a compound controller is proposed. We show the effectiveness of the compound controller by training an agent to successfully perform a recovery task involving contacts and collisions. Simulation results demonstrate that using the compound controller, the ballbot can keep balance under a larger set of initial tilting angles, compared to the conventional model-based controller.