RG-Attn: Radian Glue Attention for Multi-modality Multi-agent Cooperative Perception

TL;DR

This paper introduces RG-Attn, a novel cross-modal fusion module for multi-agent cooperative perception in autonomous driving, enabling efficient, flexible, and accurate sensor data fusion across heterogeneous modalities and agent configurations.

Contribution

We propose RG-Attn, a lightweight, transformation-based attention module that unifies intra- and inter-agent fusion, supporting diverse sensor setups and improving perception accuracy in multi-agent systems.

Findings

Achieves state-of-the-art detection accuracy on multiple datasets.

Supports flexible sensor configurations including LiDAR and cameras.

Demonstrates high efficiency and robustness in real-world scenarios.

Abstract

Cooperative perception enhances autonomous driving by leveraging Vehicle-to-Everything (V2X) communication for multi-agent sensor fusion. However, most existing methods rely on single-modal data sharing, limiting fusion performance, particularly in heterogeneous sensor settings involving both LiDAR and cameras across vehicles and roadside units (RSUs). To address this, we propose Radian Glue Attention (RG-Attn), a lightweight and generalizable cross-modal fusion module that unifies intra-agent and inter-agent fusion via transformation-based coordinate alignment and a unified sampling/inversion strategy. RG-Attn efficiently aligns features through a radian-based attention constraint, operating column-wise on geometrically consistent regions to reduce overhead and preserve spatial coherence, thereby enabling accurate and robust fusion. Building upon RG-Attn, we propose three cooperative architectures. The first, Paint-To-Puzzle (PTP), prioritizes communication efficiency but assumes all agents have LiDAR, optionally paired with cameras. The second, Co-Sketching-Co-Coloring (CoS-CoCo), offers maximal flexibility, supporting any sensor setup (e.g., LiDAR-only, camera-only, or both) and enabling strong cross-modal generalization for real-world deployment. The third, Pyramid-RG-Attn Fusion (PRGAF), aims for peak detection accuracy with the highest computational overhead. Extensive evaluations on simulated and real-world datasets show our framework delivers state-of-the-art detection accuracy with high flexibility and efficiency. GitHub Link: https://github.com/LantaoLi/RG-Attn