EagleVision: A Multi-Task Benchmark for Cross-Domain Perception in High-Speed Autonomous Racing

TL;DR

EagleVision introduces a comprehensive multi-task benchmark for high-speed autonomous racing perception, enabling systematic evaluation of cross-domain generalization in LiDAR-based detection and trajectory prediction.

Contribution

It provides a unified, annotated dataset and evaluation protocol for high-speed racing perception tasks, facilitating research on domain transfer and generalization.

Findings

Pretraining on urban data improves detection performance.

Intermediate pretraining on real racing data enhances transfer to racing domain.

Models trained on Indy data outperform in-domain models on trajectory prediction.

Abstract

High-speed autonomous racing presents extreme perception challenges, including large relative velocities and substantial domain shifts from conventional urban-driving datasets. Existing benchmarks do not adequately capture these high-dynamic conditions. We introduce EagleVision, a unified LiDAR-based multi-task benchmark for 3D detection and trajectory prediction in high-speed racing, providing newly annotated 3D bounding boxes for the Indy Autonomous Challenge dataset (14,893 frames) and the A2RL Real competition dataset (1,163 frames), together with 12,000 simulator-generated annotated frames, all standardized under a common evaluation protocol. Using a dataset-centric transfer framework, we quantify cross-domain generalization across urban, simulator, and real racing domains. Urban pretraining improves detection over scratch training (NDS 0.72 vs. 0.69), while intermediate pretraining on real racing data achieves the best transfer to A2RL (NDS 0.726), outperforming simulator-only adaptation. For trajectory prediction, Indy-trained models surpass in-domain A2RL training on A2RL test sequences (FDE 0.947 vs. 1.250), highlighting the role of motion-distribution coverage in cross-domain forecasting. EagleVision enables systematic study of perception generalization under extreme high-speed dynamics. The dataset and benchmark are publicly available at https://avlab.io/EagleVision