End-to-End Deep Learning for Steering Autonomous Vehicles Considering Temporal Dependencies

TL;DR

This paper introduces a novel end-to-end deep learning model that incorporates temporal dependencies for autonomous vehicle steering, significantly improving accuracy and stability over existing methods.

Contribution

The work presents a C-LSTM model that captures both visual and temporal information and reformulates steering as a classification problem with a sinusoidal output encoding.

Findings

Steering RMSE reduced by 35%

Steering stability improved by 87%

Effective use of temporal dependencies in driving models

Abstract

Steering a car through traffic is a complex task that is difficult to cast into algorithms. Therefore, researchers turn to training artificial neural networks from front-facing camera data stream along with the associated steering angles. Nevertheless, most existing solutions consider only the visual camera frames as input, thus ignoring the temporal relationship between frames. In this work, we propose a Convolutional Long Short-Term Memory Recurrent Neural Network (C-LSTM), that is end-to-end trainable, to learn both visual and dynamic temporal dependencies of driving. Additionally, We introduce posing the steering angle regression problem as classification while imposing a spatial relationship between the output layer neurons. Such method is based on learning a sinusoidal function that encodes steering angles. To train and validate our proposed methods, we used the publicly available Comma.ai dataset. Our solution improved steering root mean square error by 35% over recent methods, and led to a more stable steering by 87%.