Deeply learned face representations are sparse, selective, and robust

TL;DR

This paper introduces DeepID2+, a deep face recognition network that exhibits sparsity, selectiveness, and robustness, achieving state-of-the-art results by leveraging high-dimensional representations and supervision at multiple layers.

Contribution

The paper demonstrates that neural activations are moderately sparse, highly selective to identities and attributes, and robust to occlusions, revealing critical properties for high-performance face recognition.

Findings

Achieves state-of-the-art on LFW and YouTube Faces benchmarks.

Neural responses remain effective even when binarized.

Higher-layer neurons are highly selective to identities and attributes.

Abstract

This paper designs a high-performance deep convolutional network (DeepID2+) for face recognition. It is learned with the identification-verification supervisory signal. By increasing the dimension of hidden representations and adding supervision to early convolutional layers, DeepID2+ achieves new state-of-the-art on LFW and YouTube Faces benchmarks. Through empirical studies, we have discovered three properties of its deep neural activations critical for the high performance: sparsity, selectiveness and robustness. (1) It is observed that neural activations are moderately sparse. Moderate sparsity maximizes the discriminative power of the deep net as well as the distance between images. It is surprising that DeepID2+ still can achieve high recognition accuracy even after the neural responses are binarized. (2) Its neurons in higher layers are highly selective to identities and identity-related attributes. We can identify different subsets of neurons which are either constantly excited or inhibited when different identities or attributes are present. Although DeepID2+ is not taught to distinguish attributes during training, it has implicitly learned such high-level concepts. (3) It is much more robust to occlusions, although occlusion patterns are not included in the training set.