Dodging Attack Using Carefully Crafted Natural Makeup

TL;DR

This paper introduces a novel black-box adversarial attack using carefully crafted natural makeup to evade face recognition systems in real-world scenarios, significantly reducing identification rates without raising suspicion.

Contribution

The study presents a new physical domain AML attack employing natural makeup, demonstrating its effectiveness against face recognition models in real-world conditions.

Findings

Digital domain: all participants evaded recognition

Physical domain: only 1.22% frames identified

Compared to 47.57% without makeup

Abstract

Deep learning face recognition models are used by state-of-the-art surveillance systems to identify individuals passing through public areas (e.g., airports). Previous studies have demonstrated the use of adversarial machine learning (AML) attacks to successfully evade identification by such systems, both in the digital and physical domains. Attacks in the physical domain, however, require significant manipulation to the human participant's face, which can raise suspicion by human observers (e.g. airport security officers). In this study, we present a novel black-box AML attack which carefully crafts natural makeup, which, when applied on a human participant, prevents the participant from being identified by facial recognition models. We evaluated our proposed attack against the ArcFace face recognition model, with 20 participants in a real-world setup that includes two cameras, different shooting angles, and different lighting conditions. The evaluation results show that in the digital domain, the face recognition system was unable to identify all of the participants, while in the physical domain, the face recognition system was able to identify the participants in only 1.22% of the frames (compared to 47.57% without makeup and 33.73% with random natural makeup), which is below a reasonable threshold of a realistic operational environment.