PEFT-DML: Parameter-Efficient Fine-Tuning Deep Metric Learning for Robust Multi-Modal 3D Object Detection in Autonomous Driving

TL;DR

PEFT-DML introduces a parameter-efficient deep metric learning framework that enhances multi-modal 3D object detection robustness in autonomous driving by mapping diverse sensor data into a shared space, handling sensor dropout and domain shifts effectively.

Contribution

It proposes a novel, efficient framework combining LoRA and adapters for robust multi-modal 3D detection, accommodating sensor variability and improving training efficiency.

Findings

Outperforms existing methods on nuScenes benchmark.

Maintains detection accuracy under sensor dropout.

Enhances robustness to weather and domain shifts.

Abstract

This study introduces PEFT-DML, a parameter-efficient deep metric learning framework for robust multi-modal 3D object detection in autonomous driving. Unlike conventional models that assume fixed sensor availability, PEFT-DML maps diverse modalities (LiDAR, radar, camera, IMU, GNSS) into a shared latent space, enabling reliable detection even under sensor dropout or unseen modality class combinations. By integrating Low-Rank Adaptation (LoRA) and adapter layers, PEFT-DML achieves significant training efficiency while enhancing robustness to fast motion, weather variability, and domain shifts. Experiments on benchmarks nuScenes demonstrate superior accuracy.