CooTest: An Automated Testing Approach for V2X Communication Systems

TL;DR

CooTest is an automated testing tool designed for V2X cooperative perception systems in autonomous vehicles, enhancing detection accuracy and reliability by generating transformed driving scenes to identify errors.

Contribution

This paper introduces CooTest, the first automated testing framework specifically for V2X cooperative perception modules, addressing challenges of interpretability and data collection.

Findings

CooTest effectively detects erroneous behaviors in cooperative perception models.

It improves detection average precision in various driving conditions.

CooTest reduces misleading cooperation errors through retraining with generated scenes.

Abstract

Perceiving the complex driving environment precisely is crucial to the safe operation of autonomous vehicles. With the tremendous advancement of deep learning and communication technology, Vehicle-to-Everything (V2X) collaboration has the potential to address limitations in sensing distant objects and occlusion for a single-agent perception system. However, despite spectacular progress, several communication challenges can undermine the effectiveness of multi-vehicle cooperative perception. The low interpretability of Deep Neural Networks (DNNs) and the high complexity of communication mechanisms make conventional testing techniques inapplicable for the cooperative perception of autonomous driving systems (ADS). Besides, the existing testing techniques, depending on manual data collection and labeling, become time-consuming and prohibitively expensive. In this paper, we design and implement CooTest, the first automated testing tool of the V2X-oriented cooperative perception module. CooTest devises the V2X-specific metamorphic relation and equips communication and weather transformation operators that can reflect the impact of the various cooperative driving factors to produce transformed scenes. Furthermore, we adopt a V2X-oriented guidance strategy for the transformed scene generation process and improve testing efficiency. We experiment CooTest with multiple cooperative perception models with different fusion schemes to evaluate its performance on different tasks. The experiment results show that CooTest can effectively detect erroneous behaviors under various V2X-oriented driving conditions. Also, the results confirm that CooTest can improve detection average precision and decrease misleading cooperation errors by retraining with the generated scenes.