End to End Vehicle Lateral Control Using a Single Fisheye Camera

TL;DR

This paper introduces a novel end-to-end vehicle lateral control system using a single fisheye camera, with a custom simulator for data augmentation and evaluation, achieving over 99% autonomy in urban driving scenarios.

Contribution

The paper presents a new model and simulator for vehicle control using only a fisheye camera, reducing reliance on multiple long-range cameras.

Findings

Achieved over 99% autonomy in urban driving in simulation.

Successfully tested on real-world scenarios with obstacle avoidance.

Demonstrated effective data augmentation with a single fisheye camera.

Abstract

Convolutional neural networks are commonly used to control the steering angle for autonomous cars. Most of the time, multiple long range cameras are used to generate lateral failure cases. In this paper we present a novel model to generate this data and label augmentation using only one short range fisheye camera. We present our simulator and how it can be used as a consistent metric for lateral end-to-end control evaluation. Experiments are conducted on a custom dataset corresponding to more than 10000 km and 200 hours of open road driving. Finally we evaluate this model on real world driving scenarios, open road and a custom test track with challenging obstacle avoidance and sharp turns. In our simulator based on real-world videos, the final model was capable of more than 99% autonomy on urban road