Adversarial Attacks on Camera-LiDAR Models for 3D Car Detection

TL;DR

This paper investigates the vulnerability of multi-modal 3D car detection models in autonomous vehicles to physically realizable adversarial attacks, demonstrating significant evasion success on KITTI benchmark.

Contribution

It introduces a universal adversarial object attack applicable to both cascaded and fusion perception models, highlighting their vulnerabilities in a multi-modal setting.

Findings

Adversarial objects caused over 50% detection evasion.

RGB input played a larger role in attack success.

Both model types are vulnerable to the proposed attack.

Abstract

Most autonomous vehicles (AVs) rely on LiDAR and RGB camera sensors for perception. Using these point cloud and image data, perception models based on deep neural nets (DNNs) have achieved state-of-the-art performance in 3D detection. The vulnerability of DNNs to adversarial attacks has been heavily investigated in the RGB image domain and more recently in the point cloud domain, but rarely in both domains simultaneously. Multi-modal perception systems used in AVs can be divided into two broad types: cascaded models which use each modality independently, and fusion models which learn from different modalities simultaneously. We propose a universal and physically realizable adversarial attack for each type, and study and contrast their respective vulnerabilities to attacks. We place a single adversarial object with specific shape and texture on top of a car with the objective of making this car evade detection. Evaluating on the popular KITTI benchmark, our adversarial object made the host vehicle escape detection by each model type more than 50% of the time. The dense RGB input contributed more to the success of the adversarial attacks on both cascaded and fusion models.