Lateral Ego-Vehicle Control without Supervision using Point Clouds

TL;DR

This paper introduces a robust, scalable lateral vehicle control framework that uses unlabeled RGB sequences and point clouds, outperforming supervised models in real-world scenarios.

Contribution

It presents a novel training framework that leverages unlabeled RGB data and point clouds for lateral vehicle control, enhancing robustness without supervision.

Findings

Outperforms supervised models in online experiments.

Uses only unlabeled RGB sequences for training.

Generates additional trajectories to improve robustness.

Abstract

Existing vision based supervised approaches to lateral vehicle control are capable of directly mapping RGB images to the appropriate steering commands. However, they are prone to suffering from inadequate robustness in real world scenarios due to a lack of failure cases in the training data. In this paper, a framework for training a more robust and scalable model for lateral vehicle control is proposed. The framework only requires an unlabeled sequence of RGB images. The trained model takes a point cloud as input and predicts the lateral offset to a subsequent frame from which the steering angle is inferred. The frame poses are in turn obtained from visual odometry. The point cloud is conceived by projecting dense depth maps into 3D. An arbitrary number of additional trajectories from this point cloud can be generated during training. This is to increase the robustness of the model. Online experiments show that the performance of our method is superior to that of the supervised model.