Real-World Adversarial Examples involving Makeup Application

TL;DR

This paper introduces a novel physical adversarial attack on face recognition systems using full-face makeup generated by cycle-GAN, demonstrating its effectiveness and robustness against manual application errors.

Contribution

It presents a new physical adversarial attack method employing makeup and cycle-GAN, highlighting its impact on face recognition security and the influence of training data.

Findings

The attack effectively fools VGG 16-based face recognition.

Adversarial makeup remains effective despite manual application errors.

Training data influences the success of physical adversarial attacks.

Abstract

Deep neural networks have developed rapidly and have achieved outstanding performance in several tasks, such as image classification and natural language processing. However, recent studies have indicated that both digital and physical adversarial examples can fool neural networks. Face-recognition systems are used in various applications that involve security threats from physical adversarial examples. Herein, we propose a physical adversarial attack with the use of full-face makeup. The presence of makeup on the human face is a reasonable possibility, which possibly increases the imperceptibility of attacks. In our attack framework, we combine the cycle-adversarial generative network (cycle-GAN) and a victimized classifier. The Cycle-GAN is used to generate adversarial makeup, and the architecture of the victimized classifier is VGG 16. Our experimental results show that our attack can effectively overcome manual errors in makeup application, such as color and position-related errors. We also demonstrate that the approaches used to train the models can influence physical attacks; the adversarial perturbations crafted from the pre-trained model are affected by the corresponding training data.