Scaling Vision-based End-to-End Driving with Multi-View Attention Learning

TL;DR

This paper introduces CIL++, an improved vision-based end-to-end driving model that uses high-resolution images and attention mechanisms, achieving competitive performance with less costly supervision.

Contribution

CIL++ enhances CILRS by processing higher-resolution images with a human-inspired HFOV and integrating a proper attention mechanism, serving as a strong, cost-effective baseline.

Findings

CIL++ achieves performance comparable to more expensive models.

Using high-resolution images and attention improves vision-based driving.

CIL++ is a cost-effective alternative for end-to-end driving models.

Abstract

On end-to-end driving, human driving demonstrations are used to train perception-based driving models by imitation learning. This process is supervised on vehicle signals (e.g., steering angle, acceleration) but does not require extra costly supervision (human labeling of sensor data). As a representative of such vision-based end-to-end driving models, CILRS is commonly used as a baseline to compare with new driving models. So far, some latest models achieve better performance than CILRS by using expensive sensor suites and/or by using large amounts of human-labeled data for training. Given the difference in performance, one may think that it is not worth pursuing vision-based pure end-to-end driving. However, we argue that this approach still has great value and potential considering cost and maintenance. In this paper, we present CIL++, which improves on CILRS by both processing higher-resolution images using a human-inspired HFOV as an inductive bias and incorporating a proper attention mechanism. CIL++ achieves competitive performance compared to models which are more costly to develop. We propose to replace CILRS with CIL++ as a strong vision-based pure end-to-end driving baseline supervised by only vehicle signals and trained by conditional imitation learning.