Suspension Regulation of Medium-low-speed Maglev Trains via Deep Reinforcement Learning

TL;DR

This paper introduces a model-free reinforcement learning approach for suspension regulation in medium-low-speed maglev trains, effectively handling system uncertainties and disturbances, and outperforming traditional PID controllers.

Contribution

The paper develops a novel RL-based suspension controller using neural networks and double Q-learning, demonstrating superior performance over PID and comparable results to model-based methods.

Findings

RL controllers outperform PID in real data tests

Proposed method achieves comparable results to model-based controllers

Double Q-learning enhances regulation performance

Abstract

The suspension regulation is critical to the operation of medium-low-speed maglev trains (mlsMTs). Due to uncertain environment, strong disturbances and high nonlinearity of the system dynamics, this problem cannot be well solved by most of the model-based controllers. In this paper, we propose a model-free controller by reformulating it as a continuous-state, continuous-action Markov decision process (MDP) with unknown transition probabilities. With the deterministic policy gradient and neural network approximation, we design reinforcement learning (RL) algorithms to solve the MDP and obtain a state-feedback controller by using sampled data from the suspension system. To further improve its performance, we adopt a double Q-learning scheme for learning the regulation controller. We illustrate that the proposed controllers outperform the existing PID controller with a real dataset from the mlsMT in Changsha, China and is even comparable to model-based controllers, which assume that the complete information of the model is known, via simulations.