ModalPatch: A Plug-and-Play Module for Robust Multi-Modal 3D Object Detection under Modality Drop
Shuangzhi Li, Lei Ma, and Xingyu Li

TL;DR
ModalPatch is a versatile plug-and-play module that enhances the robustness of multi-modal 3D object detection systems against transient modality drops without requiring retraining or architectural modifications.
Contribution
It introduces ModalPatch, a novel, easy-to-integrate module that leverages temporal data and uncertainty-guided fusion to maintain detection performance during modality interruptions.
Findings
Significantly improves detection robustness under modality drop scenarios.
Does not require retraining or architectural changes to existing detectors.
Enhances both robustness and accuracy across multiple detection frameworks.
Abstract
Multi-modal 3D object detection is pivotal for autonomous driving, integrating complementary sensors like LiDAR and cameras. However, its real-world reliability is challenged by transient data interruptions and missing, where modalities can momentarily drop due to hardware glitches, adverse weather, or occlusions. This poses a critical risk, especially during a simultaneous modality drop, where the vehicle is momentarily blind. To address this problem, we introduce ModalPatch, the first plug-and-play module designed to enable robust detection under arbitrary modality-drop scenarios. Without requiring architectural changes or retraining, ModalPatch can be seamlessly integrated into diverse detection frameworks. Technically, ModalPatch leverages the temporal nature of sensor data for perceptual continuity, using a history-based module to predict and compensate for transiently unavailable…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Neural Network Applications · Autonomous Vehicle Technology and Safety · Advanced Optical Sensing Technologies
