End-to-End Autonomous Driving through V2X Cooperation

TL;DR

This paper introduces UniV2X, an end-to-end cooperative autonomous driving framework that integrates sensor data from vehicles and infrastructure via V2X communication, significantly improving planning and perception performance.

Contribution

UniV2X is the first unified end-to-end framework combining all driving modules with a hybrid data transmission mechanism for vehicle-infrastructure cooperation.

Findings

Significant improvement in planning performance on DAIR-V2X dataset

Enhanced perception, mapping, and occupancy prediction results

Effective data fusion with interpretability under limited communication conditions

Abstract

Cooperatively utilizing both ego-vehicle and infrastructure sensor data via V2X communication has emerged as a promising approach for advanced autonomous driving. However, current research mainly focuses on improving individual modules, rather than taking end-to-end learning to optimize final planning performance, resulting in underutilized data potential. In this paper, we introduce UniV2X, a pioneering cooperative autonomous driving framework that seamlessly integrates all key driving modules across diverse views into a unified network. We propose a sparse-dense hybrid data transmission and fusion mechanism for effective vehicle-infrastructure cooperation, offering three advantages: 1) Effective for simultaneously enhancing agent perception, online mapping, and occupancy prediction, ultimately improving planning performance. 2) Transmission-friendly for practical and limited communication conditions. 3) Reliable data fusion with interpretability of this hybrid data. We implement UniV2X, as well as reproducing several benchmark methods, on the challenging DAIR-V2X, the real-world cooperative driving dataset. Experimental results demonstrate the effectiveness of UniV2X in significantly enhancing planning performance, as well as all intermediate output performance. The project is available at \href{https://github.com/AIR-THU/UniV2X}{https://github.com/AIR-THU/UniV2X}.