Face Identification with Second-Order Pooling

TL;DR

This paper introduces a novel face recognition method using second-order pooling of local features encoded by a small dictionary, significantly improving accuracy over previous approaches, especially on the LFW dataset.

Contribution

The work adapts second-order pooling for face recognition with a small dictionary and pooling of encoded features, achieving superior performance.

Findings

Outperforms state-of-the-art on LFW by ~13% accuracy

Highlights importance of pooling encoded features

Uses simple linear classifier for high accuracy

Abstract

Automatic face recognition has received significant performance improvement by developing specialised facial image representations. On the other hand, generic object recognition has rarely been applied to the face recognition. Spatial pyramid pooling of features encoded by an over-complete dictionary has been the key component of many state-of-the-art image classification systems. Inspired by its success, in this work we develop a new face image representation method inspired by the second-order pooling in Carreira et al. [1], which was originally proposed for image segmentation. The proposed method differs from the previous methods in that, we encode the densely extracted local patches by a small-size dictionary; and the facial image signatures are obtained by pooling the second-order statistics of the encoded features. We show the importance of pooling on encoded features, which is bypassed by the original second-order pooling method to avoid the high computational cost. Equipped with a simple linear classifier, the proposed method outperforms the state-of-the-art face identification performance by large margins. For example, on the LFW databases, the proposed method performs better than the previous best by around 13% accuracy.