Understanding the Risks of Asphalt Art to the Reliability of Vision-Based Perception Systems

TL;DR

This paper investigates how artistic asphalt crosswalks affect the performance of vision-based pedestrian detection systems, revealing that complex and adversarial designs can significantly impair detection accuracy and pose security risks.

Contribution

It introduces a realistic evaluation framework for assessing the impact of asphalt art on pedestrian detection and uncovers vulnerabilities to adversarial asphalt art patterns.

Findings

Complex asphalt art reduces pedestrian detection accuracy.

Adversarial asphalt art can obscure pedestrians or create false detections.

Simple, color-based designs have minimal impact.

Abstract

Artistic crosswalks featuring asphalt art, introduced by different organizations in recent years, aim to enhance the visibility and safety of pedestrians. However, their visual complexity may interfere with surveillance systems that rely on vision-based object detection models. In this study, we investigate the impact of asphalt art on pedestrian detection performance of a pretrained vision-based object detection model. We construct realistic crosswalk scenarios by compositing various street art patterns into a fixed surveillance scene and evaluate the model's performance in detecting pedestrians on asphalt-arted crosswalks under both benign and adversarial conditions. A benign case refers to pedestrian crosswalks painted with existing normal asphalt art, whereas an adversarial case involves digitally crafted or altered asphalt art perpetrated by an attacker. Our results show that while simple, color-based designs have minimal effect, complex artistic patterns, particularly those with high visual salience, can significantly degrade pedestrian detection performance. Furthermore, we demonstrate that adversarially crafted asphalt art can be exploited to deliberately obscure real pedestrians or generate non-existent pedestrian detections. These findings highlight a potential vulnerability in urban vision-based pedestrian surveillance systems, and underscore the importance of accounting for environmental visual variations when designing robust pedestrian perception models.