VALISENS: A Validated Innovative Multi-Sensor System for Cooperative Automated Driving

TL;DR

VALISENS is a cooperative perception system for automated driving that combines multi-sensor data from vehicles and infrastructure, improving detection range, robustness, and safety in complex environments through V2X collaboration.

Contribution

It introduces a validated multi-sensor fusion framework leveraging vehicle-to-everything communication, integrating roadside sensors and onboard sensors for enhanced perception and sensor health monitoring.

Findings

Improves pedestrian detection by up to 18% over vehicle-only sensing.

Achieves over 97% accuracy in sensor health monitoring.

Demonstrates effective real-world deployment in a dedicated testbed.

Abstract

Reliable perception remains a key challenge for Connected Automated Vehicles (CAVs) in complex real-world environments, where varying lighting conditions and adverse weather degrade sensing performance. While existing multi-sensor solutions improve local robustness, they remain constrained by limited sensing range, line-of-sight occlusions, and sensor failures on individual vehicles. This paper introduces VALISENS, a validated cooperative perception system that extends multi-sensor fusion beyond a single vehicle through Vehicle-to-Everything (V2X)-enabled collaboration between Connected Automated Vehicles (CAVs) and intelligent infrastructure. VALISENS integrates onboard and roadside LiDARs, radars, RGB cameras, and thermal cameras within a unified multi-agent perception framework. Thermal cameras enhances the detection of Vulnerable Road Users (VRUs) under challenging lighting conditions, while roadside sensors reduce occlusions and expand the effective perception range. In addition, an integrated sensor monitoring module continuously assesses sensor health and detects anomalies before system degradation occurs. The proposed system is implemented and evaluated in a dedicated real-world testbed. Experimental results show that VALISENS improves pedestrian situational awareness by up to 18% compared with vehicle-only sensing, while the sensor monitoring module achieves over 97% accuracy, demonstrating its effectiveness and its potential to support future Cooperative Intelligent Transport Systems (C-ITS) applications.