Select2Col: Leveraging Spatial-Temporal Importance of Semantic Information for Efficient Collaborative Perception

TL;DR

Select2Col introduces a novel collaborative perception framework that considers both spatial and temporal importance of semantic information, utilizing GNN-based collaborator selection and a hybrid attention fusion method to enhance perception accuracy.

Contribution

The paper proposes a new framework that incorporates spatial-temporal importance of semantic info and a GNN-based collaborator selection and fusion algorithm, improving collaborative perception.

Findings

Significantly outperforms existing methods on three datasets.

Effectively identifies contributive collaborators using GNN.

Enhances perception accuracy through spatial-temporal information fusion.

Abstract

Collaborative perception by leveraging the shared semantic information plays a crucial role in overcoming the individual limitations of isolated agents. However, existing collaborative perception methods tend to focus solely on the spatial features of semantic information, while neglecting the importance of the temporal dimension. Consequently, the potential benefits of collaboration remain underutilized. In this article, we propose Select2Col, a novel collaborative perception framework that takes into account the \underline{s}patial-t\underline{e}mpora\underline{l} importanc\underline{e} of semanti\underline{c} informa\underline{t}ion. Within the Select2Col, we develop a collaborator selection method that utilizes a lightweight graph neural network (GNN) to estimate the importance of semantic information (IoSI) of each collaborator in enhancing perception performance, thereby identifying contributive collaborators while excluding those that potentially bring negative impact. Moreover, we present a semantic information fusion algorithm called HPHA (historical prior hybrid attention), which integrates multi-scale attention and short-term attention modules to capture the IoSI in feature representation from the spatial and temporal dimensions respectively, and assigns IoSI-consistent weights for efficient fusion of information from selected collaborators. Extensive experiments on three open datasets demonstrate that our proposed Select2Col significantly improves the perception performance compared to state-of-the-art approaches. The code associated with this research is publicly available at https://github.com/huangqzj/Select2Col/.