Bridging the Domain Gap for Multi-Agent Perception

TL;DR

This paper introduces a lightweight, plug-in framework that effectively bridges domain gaps in multi-agent perception systems with different neural network models, significantly improving detection accuracy.

Contribution

It presents the first lightweight, adaptable framework with a feature resizer and cross-domain transformer to address domain gaps in multi-agent perception.

Findings

Outperforms baseline methods by at least 8% in 3D object detection

Effectively bridges feature domain gaps in multi-agent perception

Compatible as a plug-in module for existing systems

Abstract

Existing multi-agent perception algorithms usually select to share deep neural features extracted from raw sensing data between agents, achieving a trade-off between accuracy and communication bandwidth limit. However, these methods assume all agents have identical neural networks, which might not be practical in the real world. The transmitted features can have a large domain gap when the models differ, leading to a dramatic performance drop in multi-agent perception. In this paper, we propose the first lightweight framework to bridge such domain gaps for multi-agent perception, which can be a plug-in module for most existing systems while maintaining confidentiality. Our framework consists of a learnable feature resizer to align features in multiple dimensions and a sparse cross-domain transformer for domain adaption. Extensive experiments on the public multi-agent perception dataset V2XSet have demonstrated that our method can effectively bridge the gap for features from different domains and outperform other baseline methods significantly by at least 8% for point-cloud-based 3D object detection.