Beyond Frontal Faces: Improving Person Recognition Using Multiple Cues

TL;DR

This paper introduces the PIPA dataset for unconstrained person recognition and proposes the PIPER method, which combines multiple cues to improve recognition accuracy in challenging real-world photo album scenarios.

Contribution

The paper presents a new large-scale dataset for unconstrained person recognition and a novel multi-cue recognition method that outperforms existing face recognition techniques in such settings.

Findings

PIPER significantly outperforms DeepFace on the PIPA dataset.

The dataset contains over 60,000 instances of 2,000 individuals from Flickr.

Multi-cue integration improves recognition in non-frontal, varied conditions.

Abstract

We explore the task of recognizing peoples' identities in photo albums in an unconstrained setting. To facilitate this, we introduce the new People In Photo Albums (PIPA) dataset, consisting of over 60000 instances of 2000 individuals collected from public Flickr photo albums. With only about half of the person images containing a frontal face, the recognition task is very challenging due to the large variations in pose, clothing, camera viewpoint, image resolution and illumination. We propose the Pose Invariant PErson Recognition (PIPER) method, which accumulates the cues of poselet-level person recognizers trained by deep convolutional networks to discount for the pose variations, combined with a face recognizer and a global recognizer. Experiments on three different settings confirm that in our unconstrained setup PIPER significantly improves on the performance of DeepFace, which is one of the best face recognizers as measured on the LFW dataset.