Latency-Aware Collaborative Perception

TL;DR

This paper introduces a latency-aware collaborative perception system for autonomous driving that actively compensates for communication delays, significantly improving robustness and performance over existing methods in latency-prone environments.

Contribution

The paper presents the first latency-aware collaborative perception system with a novel SyncNet module for feature synchronization under communication latency.

Findings

Outperforms state-of-the-art methods by 15.6% in latency scenarios.

Maintains superior perception compared to single agents under severe latency.

Demonstrates robustness and effectiveness of the proposed system.

Abstract

Collaborative perception has recently shown great potential to improve perception capabilities over single-agent perception. Existing collaborative perception methods usually consider an ideal communication environment. However, in practice, the communication system inevitably suffers from latency issues, causing potential performance degradation and high risks in safety-critical applications, such as autonomous driving. To mitigate the effect caused by the inevitable latency, from a machine learning perspective, we present the first latency-aware collaborative perception system, which actively adapts asynchronous perceptual features from multiple agents to the same time stamp, promoting the robustness and effectiveness of collaboration. To achieve such a feature-level synchronization, we propose a novel latency compensation module, called SyncNet, which leverages feature-attention symbiotic estimation and time modulation techniques. Experiments results show that the proposed latency aware collaborative perception system with SyncNet can outperforms the state-of-the-art collaborative perception method by 15.6% in the communication latency scenario and keep collaborative perception being superior to single agent perception under severe latency.