LiDAR-based End-to-end Temporal Perception for Vehicle-Infrastructure Cooperation

TL;DR

This paper introduces LET-VIC, a LiDAR-based end-to-end framework for vehicle-infrastructure cooperation that enhances temporal perception and tracking accuracy through novel attention modules and calibration error compensation.

Contribution

The paper presents a novel end-to-end LiDAR-based framework with attention modules and calibration error compensation for improved vehicle-infrastructure cooperative perception.

Findings

Achieves +15.0% mAP improvement over baseline

Surpasses existing models like V2VNet and FFNet in detection and tracking

Demonstrates robustness without considering communication delays

Abstract

Temporal perception, defined as the capability to detect and track objects across temporal sequences, serves as a fundamental component in autonomous driving systems. While single-vehicle perception systems encounter limitations, stemming from incomplete perception due to object occlusion and inherent blind spots, cooperative perception systems present their own challenges in terms of sensor calibration precision and positioning accuracy. To address these issues, we introduce LET-VIC, a LiDAR-based End-to-End Tracking framework for Vehicle-Infrastructure Cooperation (VIC). First, we employ Temporal Self-Attention and VIC Cross-Attention modules to effectively integrate temporal and spatial information from both vehicle and infrastructure perspectives. Then, we develop a novel Calibration Error Compensation (CEC) module to mitigate sensor misalignment issues and facilitate accurate feature alignment. Experiments on the V2X-Seq-SPD dataset demonstrate that LET-VIC significantly outperforms baseline models. Compared to LET-V, LET-VIC achieves +15.0% improvement in mAP and a +17.3% improvement in AMOTA. Furthermore, LET-VIC surpasses representative Tracking by Detection models, including V2VNet, FFNet, and PointPillars, with at least a +13.7% improvement in mAP and a +13.1% improvement in AMOTA without considering communication delays, showcasing its robust detection and tracking performance. The experiments demonstrate that the integration of multi-view perspectives, temporal sequences, or CEC in end-to-end training significantly improves both detection and tracking performance. All code will be open-sourced.