Multi-Prototype Networks for Unconstrained Set-based Face Recognition

TL;DR

This paper introduces Multi-Prototype Networks (MPNet) for unconstrained set-based face recognition, effectively handling large intra-set variance by learning multiple representative prototypes adaptively from media sets.

Contribution

The paper proposes a novel MPNet model with a Dense SubGraph learning component to automatically learn multiple discriminative face prototypes under varying conditions.

Findings

MPNet outperforms state-of-the-art methods in accuracy.

The Dense SubGraph component effectively untangles media inconsistencies.

Multiple prototypes improve recognition robustness.

Abstract

In this paper, we study the challenging unconstrained set-based face recognition problem where each subject face is instantiated by a set of media (images and videos) instead of a single image. Naively aggregating information from all the media within a set would suffer from the large intra-set variance caused by heterogeneous factors (e.g., varying media modalities, poses and illuminations) and fail to learn discriminative face representations. A novel Multi-Prototype Network (MPNet) model is thus proposed to learn multiple prototype face representations adaptively from the media sets. Each learned prototype is representative for the subject face under certain condition in terms of pose, illumination and media modality. Instead of handcrafting the set partition for prototype learning, MPNet introduces a Dense SubGraph (DSG) learning sub-net that implicitly untangles inconsistent media and learns a number of representative prototypes. Qualitative and quantitative experiments clearly demonstrate superiority of the proposed model over state-of-the-arts.