Fooling thermal infrared pedestrian detectors in real world using small bulbs

TL;DR

This paper demonstrates a physical attack method using small bulbs to significantly reduce the effectiveness of infrared pedestrian detectors like YOLOv3 in real-world scenarios, revealing security vulnerabilities.

Contribution

It introduces a novel physical multispectral attack with small bulbs that can fool infrared and visible pedestrian detectors in real-world environments.

Findings

Digital AP dropped by 64.12% with patches

Physical board reduced AP by 34.48% in real world

First multispectral attack hiding from infrared and visible detectors

Abstract

Thermal infrared detection systems play an important role in many areas such as night security, autonomous driving, and body temperature detection. They have the unique advantages of passive imaging, temperature sensitivity and penetration. But the security of these systems themselves has not been fully explored, which poses risks in applying these systems. We propose a physical attack method with small bulbs on a board against the state of-the-art pedestrian detectors. Our goal is to make infrared pedestrian detectors unable to detect real-world pedestrians. Towards this goal, we first showed that it is possible to use two kinds of patches to attack the infrared pedestrian detector based on YOLOv3. The average precision (AP) dropped by 64.12% in the digital world, while a blank board with the same size caused the AP to drop by 29.69% only. After that, we designed and manufactured a physical board and successfully attacked YOLOv3 in the real world. In recorded videos, the physical board caused AP of the target detector to drop by 34.48%, while a blank board with the same size caused the AP to drop by 14.91% only. With the ensemble attack techniques, the designed physical board had good transferability to unseen detectors. We also proposed the first physical multispectral (infrared and visible) attack. By using a combination method, we successfully hide from the visible light and infrared object detection systems at the same time.