Stabilization of vertical motion of a vehicle on bumpy terrain using deep reinforcement learning

TL;DR

This paper presents a deep reinforcement learning approach to stabilize the vertical motion of vehicles on bumpy terrain by controlling velocity, addressing challenges like changing vehicle parameters and off-road conditions.

Contribution

It introduces a novel deep reinforcement learning control policy that considers vehicle velocity modulation for vertical stabilization in complex terrains.

Findings

The RL agent effectively reduces vertical acceleration on bumpy terrain.

The method adapts to changes in vehicle parameters like mass and suspension stiffness.

It outperforms traditional suspension control methods in simulated scenarios.

Abstract

Stabilizing vertical dynamics for on-road and off-road vehicles is an important research area that has been looked at mostly from the point of view of ride comfort. The advent of autonomous vehicles now shifts the focus more towards developing stabilizing techniques from the point of view of onboard proprioceptive and exteroceptive sensors whose real-time measurements influence the performance of an autonomous vehicle. The current solutions to this problem of managing the vertical oscillations usually limit themselves to the realm of active suspension systems without much consideration to modulating the vehicle velocity, which plays an important role by the virtue of the fact that vertical and longitudinal dynamics of a ground vehicle are coupled. The task of stabilizing vertical oscillations for military ground vehicles becomes even more challenging due lack of structured environments, like city roads or highways, in off-road scenarios. Moreover, changes in structural parameters of the vehicle, such as mass (due to changes in vehicle loading), suspension stiffness and damping values can have significant effect on the controller's performance. This demands the need for developing deep learning based control policies, that can take into account an extremely large number of input features and approximate a near optimal control action. In this work, these problems are addressed by training a deep reinforcement learning agent to minimize the vertical acceleration of a scaled vehicle travelling over bumps by controlling its velocity.