Face Attributes as Cues for Deep Face Recognition Understanding

TL;DR

This paper investigates how deep face recognition networks implicitly learn face attributes by analyzing hidden layer outputs, revealing that these attributes are encoded almost as accurately as in dedicated attribute networks.

Contribution

It demonstrates that face attributes are implicitly encoded in deep face recognition models and identifies the specific neurons responsible for these attributes.

Findings

Gender, eyeglasses, and hat usage can be predicted with over 96% accuracy.

Latent neurons encode face attributes almost as well as dedicated attribute networks.

Semantic concepts are distributed within specific layers of the network.

Abstract

Deeply learned representations are the state-of-the-art descriptors for face recognition methods. These representations encode latent features that are difficult to explain, compromising the confidence and interpretability of their predictions. Most attempts to explain deep features are visualization techniques that are often open to interpretation. Instead of relying only on visualizations, we use the outputs of hidden layers to predict face attributes. The obtained performance is an indicator of how well the attribute is implicitly learned in that layer of the network. Using a variable selection technique, we also analyze how these semantic concepts are distributed inside each layer, establishing the precise location of relevant neurons for each attribute. According to our experiments, gender, eyeglasses and hat usage can be predicted with over 96% accuracy even when only a single neural output is used to predict each attribute. These performances are less than 3 percentage points lower than the ones achieved by deep supervised face attribute networks. In summary, our experiments show that, inside DCNNs optimized for face identification, there exists latent neurons encoding face attributes almost as accurately as DCNNs optimized for these attributes.