Simulated Adversarial Testing of Face Recognition Models

TL;DR

This paper introduces a framework for adversarial testing of face recognition models using simulators to uncover weaknesses not detected by standard validation datasets, enhancing model robustness before deployment.

Contribution

It presents a novel adversarial testing method leveraging simulators to identify model vulnerabilities in a controlled, interpretable manner, specifically applied to face recognition systems.

Findings

Simulated adversarial faces can fool face recognition models.

Weaknesses in models trained on real data are revealed through simulation.

Adversarial regions in the simulator's latent space can be systematically identified.

Abstract

Most machine learning models are validated and tested on fixed datasets. This can give an incomplete picture of the capabilities and weaknesses of the model. Such weaknesses can be revealed at test time in the real world. The risks involved in such failures can be loss of profits, loss of time or even loss of life in certain critical applications. In order to alleviate this issue, simulators can be controlled in a fine-grained manner using interpretable parameters to explore the semantic image manifold. In this work, we propose a framework for learning how to test machine learning algorithms using simulators in an adversarial manner in order to find weaknesses in the model before deploying it in critical scenarios. We apply this method in a face recognition setup. We show that certain weaknesses of models trained on real data can be discovered using simulated samples. Using our proposed method, we can find adversarial synthetic faces that fool contemporary face recognition models. This demonstrates the fact that these models have weaknesses that are not measured by commonly used validation datasets. We hypothesize that this type of adversarial examples are not isolated, but usually lie in connected spaces in the latent space of the simulator. We present a method to find these adversarial regions as opposed to the typical adversarial points found in the adversarial example literature.