Adversarial Manhole: Challenging Monocular Depth Estimation and Semantic Segmentation Models with Patch Attack

TL;DR

This paper introduces a practical patch attack using manhole cover-like patches to significantly deceive monocular depth estimation and semantic segmentation models, exposing vulnerabilities in autonomous driving perception systems.

Contribution

The paper proposes a novel physical patch attack method that effectively deceives MDE and SS models, demonstrating high success rates and physical robustness.

Findings

43% relative error in depth estimation

96% attack success rate in segmentation

Effective in physical simulations

Abstract

Monocular depth estimation (MDE) and semantic segmentation (SS) are crucial for the navigation and environmental interpretation of many autonomous driving systems. However, their vulnerability to practical adversarial attacks is a significant concern. This paper presents a novel adversarial attack using practical patches that mimic manhole covers to deceive MDE and SS models. The goal is to cause these systems to misinterpret scenes, leading to false detections of near obstacles or non-passable objects. We use Depth Planar Mapping to precisely position these patches on road surfaces, enhancing the attack's effectiveness. Our experiments show that these adversarial patches cause a 43% relative error in MDE and achieve a 96% attack success rate in SS. These patches create affected error regions over twice their size in MDE and approximately equal to their size in SS. Our studies also confirm the patch's effectiveness in physical simulations, the adaptability of the patches across different target models, and the effectiveness of our proposed modules, highlighting their practical implications.