Learning for Vehicle-to-Vehicle Cooperative Perception under Lossy Communication

TL;DR

This paper addresses the challenge of lossy communication in vehicle-to-vehicle cooperative perception by proposing a novel LC-aware feature fusion method that improves 3D object detection performance under lossy V2V communication scenarios.

Contribution

It introduces a LC-aware feature fusion framework with a Repair Network and attention modules to mitigate lossy communication effects in cooperative perception.

Findings

Significant performance improvement under lossy V2V communication.

Effective in enhancing 3D object detection accuracy.

Validated on the OPV2V dataset with digital-twin CARLA simulator.

Abstract

Deep learning has been widely used in the perception (e.g., 3D object detection) of intelligent vehicle driving. Due to the beneficial Vehicle-to-Vehicle (V2V) communication, the deep learning based features from other agents can be shared to the ego vehicle so as to improve the perception of the ego vehicle. It is named as Cooperative Perception in the V2V research, whose algorithms have been dramatically advanced recently. However, all the existing cooperative perception algorithms assume the ideal V2V communication without considering the possible lossy shared features because of the Lossy Communication (LC) which is common in the complex real-world driving scenarios. In this paper, we first study the side effect (e.g., detection performance drop) by the lossy communication in the V2V Cooperative Perception, and then we propose a novel intermediate LC-aware feature fusion method to relieve the side effect of lossy communication by a LC-aware Repair Network (LCRN) and enhance the interaction between the ego vehicle and other vehicles by a specially designed V2V Attention Module (V2VAM) including intra-vehicle attention of ego vehicle and uncertainty-aware inter-vehicle attention. The extensive experiment on the public cooperative perception dataset OPV2V (based on digital-twin CARLA simulator) demonstrates that the proposed method is quite effective for the cooperative point cloud based 3D object detection under lossy V2V communication.