Canonical Face Embeddings

TL;DR

This paper shows that different CNN-based face verification models learn nearly equivalent embeddings up to rotation, indicating a shared underlying representation despite variations in training and architecture.

Contribution

The study demonstrates that face verification CNNs produce embeddings that are related by simple transformations, revealing a common learned representation across models.

Findings

Embeddings from different CNNs can be aligned with rotation or linear transformations.

Performance drops minimally when aligning embeddings with transformations.

Models learn a shared face representation despite architectural differences.

Abstract

We present evidence that many common convolutional neural networks (CNNs) trained for face verification learn functions that are nearly equivalent under rotation. More specifically, we demonstrate that one face verification model's embeddings (i.e. last-layer activations) can be compared directly to another model's embeddings after only a rotation or linear transformation, with little performance penalty. This finding is demonstrated using IJB-C 1:1 verification across the combinations of ten modern off-the-shelf CNN-based face verification models which vary in training dataset, CNN architecture, method of angular loss calculation, or some combination of the 3. These networks achieve a mean true accept rate of 0.96 at a false accept rate of 0.01. When instead evaluating embeddings generated from two CNNs, where one CNN's embeddings are mapped with a linear transformation, the mean true accept rate drops to 0.95 using the same verification paradigm. Restricting these linear maps to only perform rotation produces a mean true accept rate of 0.91. These mappings' existence suggests that a common representation is learned by models despite variation in training or structure. We discuss the broad implications a result like this has, including an example regarding face template security.