Path Following and Stabilisation of a Bicycle Model using a Reinforcement Learning Approach

TL;DR

This paper presents a reinforcement learning approach for controlling a bicycle model to achieve path following and lateral stabilization without traditional aids, demonstrating effectiveness on complex paths and speeds.

Contribution

It introduces a novel RL-based control method for bicycle stabilization and path following using a virtual model and curriculum learning, with comprehensive evaluation.

Findings

Successfully stabilizes and follows complex paths at various speeds

RL agents outperform traditional control methods in accuracy and robustness

Analysis links RL behavior with bicycle dynamics principles

Abstract

Over the years, complex control approaches have been developed to control the motion of a bicycle. Reinforcement Learning (RL), a branch of machine learning, promises easy deployment of so-called agents. Deployed agents are increasingly considered as an alternative to controllers for mechanical systems. The present work introduces an RL approach to do path following with a virtual bicycle model while simultaneously stabilising it laterally. The bicycle, modelled as the Whipple benchmark model and using multibody system dynamics, has no stabilisation aids. The agent succeeds in both path following and stabilisation of the bicycle model exclusively by outputting steering angles, which are converted into steering torques via a PD controller. Curriculum learning is applied as a state-of-the-art training strategy. Different settings for the implemented RL framework are investigated and compared to each other. The performance of the deployed agents is evaluated using different types of paths and measurements. The ability of the deployed agents to do path following and stabilisation of the bicycle model travelling between 2m/s and 7m/s along complex paths including full circles, slalom manoeuvres, and lane changes is demonstrated. Explanatory methods for machine learning are used to analyse the functionality of a deployed agent and link the introduced RL approach with research in the field of bicycle dynamics.