DSRC: Learning Density-insensitive and Semantic-aware Collaborative Representation against Corruptions

TL;DR

This paper introduces DSRC, a novel collaborative perception method that enhances robustness against real-world corruptions by learning density-insensitive and semantic-aware representations, validated through a new benchmark and extensive experiments.

Contribution

The paper proposes DSRC, a new robust collaborative perception approach with a semantic-guided distillation framework and feature reconstruction, and introduces the first benchmark for robustness evaluation in this domain.

Findings

DSRC outperforms state-of-the-art methods in both clean and corrupted environments.

The semantic-guided distillation improves density-insensitive feature learning.

Feature-to-point cloud reconstruction enhances multi-agent fusion robustness.

Abstract

As a potential application of Vehicle-to-Everything (V2X) communication, multi-agent collaborative perception has achieved significant success in 3D object detection. While these methods have demonstrated impressive results on standard benchmarks, the robustness of such approaches in the face of complex real-world environments requires additional verification. To bridge this gap, we introduce the first comprehensive benchmark designed to evaluate the robustness of collaborative perception methods in the presence of natural corruptions typical of real-world environments. Furthermore, we propose DSRC, a robustness-enhanced collaborative perception method aiming to learn Density-insensitive and Semantic-aware collaborative Representation against Corruptions. DSRC consists of two key designs: i) a semantic-guided sparse-to-dense distillation framework, which constructs multi-view dense objects painted by ground truth bounding boxes to effectively learn density-insensitive and semantic-aware collaborative representation; ii) a feature-to-point cloud reconstruction approach to better fuse critical collaborative representation across agents. To thoroughly evaluate DSRC, we conduct extensive experiments on real-world and simulated datasets. The results demonstrate that our method outperforms SOTA collaborative perception methods in both clean and corrupted conditions. Code is available at https://github.com/Terry9a/DSRC.