mmCooper: A Multi-agent Multi-stage Communication-efficient and Collaboration-robust Cooperative Perception Framework

TL;DR

mmCooper is a novel multi-agent cooperative perception framework that improves perception accuracy and communication efficiency by multi-stage collaboration and robustness to calibration errors, validated on real-world and simulated datasets.

Contribution

It introduces a multi-stage collaboration strategy and robust information filtering to enhance perception and communication efficiency in cooperative systems.

Findings

Improves perception accuracy under bandwidth constraints.

Enhances robustness to calibration errors and misalignments.

Demonstrates effectiveness on real-world and simulated datasets.

Abstract

Collaborative perception significantly enhances individual vehicle perception performance through the exchange of sensory information among agents. However, real-world deployment faces challenges due to bandwidth constraints and inevitable calibration errors during information exchange. To address these issues, we propose mmCooper, a novel multi-agent, multi-stage, communication-efficient, and collaboration-robust cooperative perception framework. Our framework leverages a multi-stage collaboration strategy that dynamically and adaptively balances intermediate- and late-stage information to share among agents, enhancing perceptual performance while maintaining communication efficiency. To support robust collaboration despite potential misalignments and calibration errors, our framework prevents misleading low-confidence sensing information from transmission and refines the received detection results from collaborators to improve accuracy. The extensive evaluation results on both real-world and simulated datasets demonstrate the effectiveness of the mmCooper framework and its components.