Fast2comm:Collaborative perception combined with prior knowledge

TL;DR

Fast2comm is a collaborative perception framework that leverages prior knowledge and efficient feature selection to improve accuracy, bandwidth usage, and robustness against localization errors in multi-agent perception systems.

Contribution

The paper introduces a novel prior knowledge-based framework with confidence feature generation, spatial prior feature selection, and decoupled feature fusion strategies for enhanced collaborative perception.

Findings

Outperforms existing methods on real-world and simulated datasets.

Effectively reduces bandwidth while maintaining high perception accuracy.

Improves robustness to localization errors.

Abstract

Collaborative perception has the potential to significantly enhance perceptual accuracy through the sharing of complementary information among agents. However, real-world collaborative perception faces persistent challenges, particularly in balancing perception performance and bandwidth limitations, as well as coping with localization errors. To address these challenges, we propose Fast2comm, a prior knowledge-based collaborative perception framework. Specifically, (1)we propose a prior-supervised confidence feature generation method, that effectively distinguishes foreground from background by producing highly discriminative confidence features; (2)we propose GT Bounding Box-based spatial prior feature selection strategy to ensure that only the most informative prior-knowledge features are selected and shared, thereby minimizing background noise and optimizing bandwidth efficiency while enhancing adaptability to localization inaccuracies; (3)we decouple the feature fusion strategies between model training and testing phases, enabling dynamic bandwidth adaptation. To comprehensively validate our framework, we conduct extensive experiments on both real-world and simulated datasets. The results demonstrate the superior performance of our model and highlight the necessity of the proposed methods. Our code is available at https://github.com/Zhangzhengbin-TJ/Fast2comm.