Spatio-Temporal Domain Awareness for Multi-Agent Collaborative Perception

TL;DR

SCOPE is a novel multi-agent perception framework that enhances autonomous vehicle perception by effectively aggregating spatio-temporal information from multiple agents, overcoming localization errors and integrating diverse data sources.

Contribution

The paper introduces SCOPE, a comprehensive end-to-end framework that leverages spatio-temporal awareness for improved collaborative perception in autonomous driving.

Findings

SCOPE outperforms existing methods in real-world and simulated scenarios.

Effective multi-source data fusion improves perception accuracy.

Temporal and spatial aggregation components are crucial for performance.

Abstract

Multi-agent collaborative perception as a potential application for vehicle-to-everything communication could significantly improve the perception performance of autonomous vehicles over single-agent perception. However, several challenges remain in achieving pragmatic information sharing in this emerging research. In this paper, we propose SCOPE, a novel collaborative perception framework that aggregates the spatio-temporal awareness characteristics across on-road agents in an end-to-end manner. Specifically, SCOPE has three distinct strengths: i) it considers effective semantic cues of the temporal context to enhance current representations of the target agent; ii) it aggregates perceptually critical spatial information from heterogeneous agents and overcomes localization errors via multi-scale feature interactions; iii) it integrates multi-source representations of the target agent based on their complementary contributions by an adaptive fusion paradigm. To thoroughly evaluate SCOPE, we consider both real-world and simulated scenarios of collaborative 3D object detection tasks on three datasets. Extensive experiments demonstrate the superiority of our approach and the necessity of the proposed components.