CoST: Efficient Collaborative Perception From Unified Spatiotemporal Perspective

TL;DR

CoST introduces a unified spatio-temporal approach for collaborative perception, improving efficiency and accuracy by aggregating multi-agent and multi-time observations into a single space, reducing data transmission and enhancing perception in challenging scenarios.

Contribution

The paper proposes CoST, a novel method that unifies multi-agent and multi-time data fusion into a single spatio-temporal space, enhancing perception performance and transmission efficiency.

Findings

Improves perception accuracy in occluded and small sensing range scenarios.

Reduces transmission bandwidth by aggregating observations into a single space.

Compatible with existing methods, boosting their performance.

Abstract

Collaborative perception shares information among different agents and helps solving problems that individual agents may face, e.g., occlusions and small sensing range. Prior methods usually separate the multi-agent fusion and multi-time fusion into two consecutive steps. In contrast, this paper proposes an efficient collaborative perception that aggregates the observations from different agents (space) and different times into a unified spatio-temporal space simultanesouly. The unified spatio-temporal space brings two benefits, i.e., efficient feature transmission and superior feature fusion. 1) Efficient feature transmission: each static object yields a single observation in the spatial temporal space, and thus only requires transmission only once (whereas prior methods re-transmit all the object features multiple times). 2) superior feature fusion: merging the multi-agent and multi-time fusion into a unified spatial-temporal aggregation enables a more holistic perspective, thereby enhancing perception performance in challenging scenarios. Consequently, our Collaborative perception with Spatio-temporal Transformer (CoST) gains improvement in both efficiency and accuracy. Notably, CoST is not tied to any specific method and is compatible with a majority of previous methods, enhancing their accuracy while reducing the transmission bandwidth.