Facial Attributes: Accuracy and Adversarial Robustness

TL;DR

This paper evaluates the robustness of deep neural networks in extracting facial attributes, revealing high variability in adversarial vulnerability across attributes and introducing the concepts of natural adversarial samples and the fast flipping attribute technique.

Contribution

It introduces the FFA method for generating adversarial examples, analyzes attribute robustness variability, and proposes the concept of natural adversarial samples in facial attribute recognition.

Findings

FFA generates more adversarial examples than traditional methods.

Adversarial robustness varies significantly among facial attributes.

Natural adversarial samples are common and resistant to additional training.

Abstract

Facial attributes, emerging soft biometrics, must be automatically and reliably extracted from images in order to be usable in stand-alone systems. While recent methods extract facial attributes using deep neural networks (DNNs) trained on labeled facial attribute data, the robustness of deep attribute representations has not been evaluated. In this paper, we examine the representational stability of several approaches that recently advanced the state of the art on the CelebA benchmark by generating adversarial examples formed by adding small, non-random perturbations to inputs yielding altered classifications. We show that our fast flipping attribute (FFA) technique generates more adversarial examples than traditional algorithms, and that the adversarial robustness of DNNs varies highly between facial attributes. We also test the correlation of facial attributes and find that only for related attributes do the formed adversarial perturbations change the classification of others. Finally, we introduce the concept of natural adversarial samples, i.e., misclassified images where predictions can be corrected via small perturbations. We demonstrate that natural adversarial samples commonly occur and show that many of these images remain misclassified even with additional training epochs, even though their correct classification may require only a small adjustment to network parameters.