Fast and Real-time End to End Control in Autonomous Racing Cars Through Representation Learning

TL;DR

This paper introduces a fast, end-to-end autonomous racing control system that combines deep scene representation learning, trajectory optimization, and classical control, enabling real-time decision-making in complex racing scenarios.

Contribution

It presents a novel end-to-end approach that integrates unsupervised scene representation learning with trajectory planning and classical control for autonomous racing.

Findings

Effective in real-time control within CARLA simulator

Achieves robust race line prediction from camera input

Combines learning and optimization for improved accuracy

Abstract

The challenges presented in an autonomous racing situation are distinct from those faced in regular autonomous driving and require faster end-to-end algorithms and consideration of a longer horizon in determining optimal current actions keeping in mind upcoming maneuvers and situations. In this paper, we propose an end-to-end method for autonomous racing that takes in as inputs video information from an onboard camera and determines final steering and throttle control actions. We use the following split to construct such a method (1) learning a low dimensional representation of the scene, (2) pre-generating the optimal trajectory for the given scene, and (3) tracking the predicted trajectory using a classical control method. In learning a low-dimensional representation of the scene, we use intermediate representations with a novel unsupervised trajectory planner to generate expert trajectories, and hence utilize them to directly predict race lines from a given front-facing input image. Thus, the proposed algorithm employs the best of two worlds - the robustness of learning-based approaches to perception and the accuracy of optimization-based approaches for trajectory generation in an end-to-end learning-based framework. We deploy and demonstrate our framework on CARLA, a photorealistic simulator for testing self-driving cars in realistic environments.