Real-Time Forecasting of Driver-Vehicle Dynamics on 3D Roads: a Deep-Learning Framework Leveraging Bayesian Optimisation

TL;DR

This paper introduces a real-time deep-learning framework combining LSTM autoencoders and Bayesian optimization to predict driver-vehicle dynamics on complex 3D roads, addressing limitations of pose-only forecasting methods.

Contribution

It presents a self-tuning, context-aware neural network framework for joint prediction of vehicle dynamics and road geometry, suitable for real-time applications in autonomous driving.

Findings

Validated on a dynamic driving simulator with complex 3D track.

Demonstrated robustness to driver and track variations.

Achieved real-time prediction capabilities.

Abstract

Most state-of-the-art works in trajectory forecasting for automotive target predicting the pose and orientation of the agents in the scene. This represents a particularly useful problem, for instance in autonomous driving, but it does not cover a spectrum of applications in control and simulation that require information on vehicle dynamics features other than pose and orientation. Also, multi-step dynamic simulation of complex multibody models does not seem to be a viable solution for real-time long-term prediction, due to the high computational time required. To bridge this gap, we present a deep-learning framework to model and predict the evolution of the coupled driver-vehicle system dynamics jointly on a complex road geometry. It consists of two components. The first, a neural network predictor, is based on Long Short-Term Memory autoencoders and fuses the information on the road geometry and the past driver-vehicle system dynamics to produce context-aware predictions. The second, a Bayesian optimiser, is proposed to tune some significant hyperparameters of the network. These govern the network complexity, as well as the features importance. The result is a self-tunable framework with real-time applicability, which allows the user to specify the features of interest. The approach has been validated with a case study centered on motion cueing algorithms, using a dataset collected during test sessions of a non-professional driver on a dynamic driving simulator. A 3D track with complex geometry has been employed as driving environment to render the prediction task challenging. Finally, the robustness of the neural network to changes in the driver and track was investigated to set guidelines for future works.