RC-GeoCP: Geometric Consensus for Radar-Camera Collaborative Perception

TL;DR

This paper introduces RC-GeoCP, a novel framework for collaborative perception that fuses 4D radar and camera data using geometric consensus, improving scene understanding with reduced communication overhead.

Contribution

It pioneers the integration of radar and camera data in collaborative perception through geometric consensus, uncertainty-aware communication, and a shared representation framework.

Findings

Achieves state-of-the-art performance on V2X-Radar and V2X-R benchmarks.

Reduces communication overhead significantly compared to existing methods.

Establishes the first unified radar-camera collaborative perception benchmark.

Abstract

Collaborative perception (CP) enhances scene understanding through multi-agent information sharing. While LiDAR-centric systems offer precise geometry, high costs and performance degradation in adverse weather necessitate multi-modal alternatives. Despite dense visual semantics and robust spatial measurements, the synergy between cameras and 4D radar remains underexplored in collaborative settings. This work introduces RC-GeoCP, the first framework to explore the fusion of 4D radar and images in CP. To resolve misalignment caused by depth ambiguity and spatial dispersion across agents, RC-GeoCP establishes a radar-anchored geometric consensus. Specifically, Geometric Structure Rectification (GSR) aligns visual semantics with geometry derived from radar to generate spatially grounded, geometry-consistent representations. Uncertainty-Aware Communication (UAC) formulates selective transmission as a conditional entropy reduction process to prioritize informative features based on inter-agent disagreement. Finally, the Consensus-Driven Assembler (CDA) aggregates multi-agent information via shared geometric anchors to form a globally coherent representation. We establish the first unified radar-camera CP benchmark on V2X-Radar and V2X-R, demonstrating state-of-the-art performance with significantly reduced communication overhead. Code will be released soon.