MR3D-Net: Dynamic Multi-Resolution 3D Sparse Voxel Grid Fusion for LiDAR-Based Collective Perception

TL;DR

MR3D-Net introduces a dynamic multi-resolution 3D sparse voxel grid fusion method for LiDAR-based collective perception, significantly improving detection performance while drastically reducing communication bandwidth requirements.

Contribution

It presents a novel multi-resolution sparse voxel grid fusion architecture that adapts to bandwidth constraints, enhancing collective perception performance.

Findings

Achieves state-of-the-art results on OPV2V benchmark.

Reduces communication bandwidth by up to 94%.

Provides a compact and adaptive environment representation.

Abstract

The safe operation of automated vehicles depends on their ability to perceive the environment comprehensively. However, occlusion, sensor range, and environmental factors limit their perception capabilities. To overcome these limitations, collective perception enables vehicles to exchange information. However, fusing this exchanged information is a challenging task. Early fusion approaches require large amounts of bandwidth, while intermediate fusion approaches face interchangeability issues. Late fusion of shared detections is currently the only feasible approach. However, it often results in inferior performance due to information loss. To address this issue, we propose MR3D-Net, a dynamic multi-resolution 3D sparse voxel grid fusion backbone architecture for LiDAR-based collective perception. We show that sparse voxel grids at varying resolutions provide a meaningful and compact environment representation that can adapt to the communication bandwidth. MR3D-Net achieves state-of-the-art performance on the OPV2V 3D object detection benchmark while reducing the required bandwidth by up to 94% compared to early fusion. Code is available at https://github.com/ekut-es/MR3D-Net