Robust Fusion of Object-Level V2X for Learned 3D Object Detection

TL;DR

This paper explores integrating vehicle-to-everything (V2X) communication data into 3D object detection for autonomous driving, emphasizing robustness to real-world V2X imperfections.

Contribution

It introduces a noise-aware training strategy and confidence encoding to improve the robustness of V2X-enhanced 3D detection models.

Findings

V2X data significantly improves detection performance under ideal conditions.

Models trained without considering noise become fragile under realistic V2X imperfections.

The proposed methods maintain high detection accuracy even with severe noise and low V2X penetration.

Abstract

Perception for automated driving is largely based on onboard environmental sensors, such as cameras and radar, which are cost-effective but limited by line-of-sight and field-of-view constraints. These inherent limitations may cause onboard perception to fail under occlusions or poor visibility conditions. In parallel, cooperative awareness via vehicle-to-everything (V2X) communication is becoming increasingly available, enabling vehicles and infrastructure to share their own state as object-level information that complements onboard perception. In this work, we study how such V2X information can be integrated into 3D object detection and how robust the resulting system is to realistic V2X imperfections. Using the nuScenes dataset, we emulate object-level cooperative awareness messages from ground truth, injecting controlled noise and object dropout to mimic real-world conditions such as latency, localization errors, and low V2X penetration rates. We convert these messages into a dedicated bird's-eye view (BEV) input and fuse them into a BEVFusion-style detector. Our results demonstrate that while object-level cooperative information can substantially improve detection performance, achieving an NDS of 0.80 under favorable conditions, models trained on idealized data become fragile and over-reliant on V2X. Conversely, our proposed noise-aware training strategy, coupled with explicit confidence encoding, enhances robustness, maintaining performance gains even under severe noise and reduced V2X penetration.