Reinforcement Learning for Elliptical Cylinder Motion Control Tasks

TL;DR

This paper explores using reinforcement learning to control the motion of an elliptical cylinder with limited torque, comparing it to classical control methods and analyzing its effectiveness in various scenarios.

Contribution

It introduces a reinforcement learning approach for elliptical cylinder control under torque constraints and compares it with traditional energy-shaping and LQR methods.

Findings

Reinforcement learning can effectively control the elliptical cylinder.

Classical controllers serve as strong baselines for comparison.

Control difficulty increases with mass and shape asymmetry.

Abstract

The control of devices with limited input always bring attention to solve by research due to its difficulty and non-trival solution. For instance, the inverted pendulum is benchmarking problem in control theory and machine learning. In this work, we are focused on the elliptical cylinder and its motion under limited torque. The inspiration of the problem is from untethered magnetic devices, which due to distance have to operate with limited input torque. In this work, the main goal is to define the control problem of elliptic cylinder with limited input torque and solve it by Reinforcement Learning. As a classical baseline, we evaluate a two-stage controller composed of an energy-shaping swing-up law and a local Linear Quadratic Regulator (LQR) stabilizer around the target equilibrium. The swing-up controller increases the system's mechanical energy to drive the state toward a neighborhood of the desired equilibrium, a linearization of the nonlinear model yields an LQR that regulates the angle and angular-rate states to the target orientation with bounded input. This swing-up + LQR policy is a strong, interpretable reference for underactuated system and serves a point of comparison to the learned policy under identical limits and parameters. The solution shows that the learning is possible however, the different cases like stabilization in upward position or rotating of half turn are very difficult for increasing mass or ellipses with a strongly unequal perimeter ratio.