Rapid Autonomous Car Control based on Spatial and Temporal Visual Cues

TL;DR

This paper introduces a deep learning-based autonomous car control system combining CNNs and LSTMs to predict steering and throttle, enabling a robot to race autonomously on various tracks.

Contribution

It presents an end-to-end deep learning approach using CNNs and LSTMs for real-time autonomous vehicle control, tested on a robot with sensor integration.

Findings

Accurately predicts steering and throttle angles in real-time

Enables autonomous racing on diverse tracks with sufficient training

Demonstrates effective sensor-based localization and control

Abstract

We present a novel approach to modern car control utilizing a combination of Deep Convolutional Neural Networks and Long Short-Term Memory Systems: Both of which are a subsection of Hierarchical Representations Learning, more commonly known as Deep Learning. Using Deep Convolutional Neural Networks and Long Short-Term Memory Systems (DCNN/LSTM), we propose an end-to-end approach to accurately predict steering angles and throttle values. We use this algorithm on our latest robot, El Toro Grande 1 (ETG) which is equipped with a variety of sensors in order to localize itself in its environment. Using previous training data and the data that it collects during circuit and drag races, it predicts throttle and steering angles in order to stay on path and avoid colliding into other robots. This allows ETG to theoretically race on any track with sufficient training data.