Robust Collaborative Perception without External Localization and Clock Devices

TL;DR

This paper introduces FreeAlign, a novel system for collaborative perception that aligns multiple agents without external localization or clock signals by recognizing geometric patterns in perceptual data.

Contribution

The work presents a new approach for spatial-temporal alignment in collaborative perception that eliminates reliance on external hardware by using graph neural networks to identify common structures.

Findings

Performs comparably to hardware-dependent systems in real-world tests

Operates robustly without external localization or clock signals

Validated on both real-world and simulated datasets

Abstract

A consistent spatial-temporal coordination across multiple agents is fundamental for collaborative perception, which seeks to improve perception abilities through information exchange among agents. To achieve this spatial-temporal alignment, traditional methods depend on external devices to provide localization and clock signals. However, hardware-generated signals could be vulnerable to noise and potentially malicious attack, jeopardizing the precision of spatial-temporal alignment. Rather than relying on external hardwares, this work proposes a novel approach: aligning by recognizing the inherent geometric patterns within the perceptual data of various agents. Following this spirit, we propose a robust collaborative perception system that operates independently of external localization and clock devices. The key module of our system,~\emph{FreeAlign}, constructs a salient object graph for each agent based on its detected boxes and uses a graph neural network to identify common subgraphs between agents, leading to accurate relative pose and time. We validate \emph{FreeAlign} on both real-world and simulated datasets. The results show that, the ~\emph{FreeAlign} empowered robust collaborative perception system perform comparably to systems relying on precise localization and clock devices.