Unique Faces Recognition in Videos

TL;DR

This paper compares various neural network architectures and loss functions for face recognition in videos, finding that 3D CNNs and 2D LSTMs with triplet loss outperform other models in top-n retrievals, especially when combined with SVMs.

Contribution

The study systematically evaluates the effectiveness of different architectures and loss functions for video face recognition, highlighting the superiority of 3D CNNs and 2D LSTMs with triplet loss.

Findings

3D CNNs and 2D LSTMs with contrastive loss do not outperform Inception with contrastive loss.

3D CNNs and 2D LSTMs with triplet loss outperform Inception with triplet loss.

Feature representations from 2D LSTM with triplet loss are most effective for facial identification.

Abstract

This paper tackles face recognition in videos employing metric learning methods and similarity ranking models. The paper compares the use of the Siamese network with contrastive loss and Triplet Network with triplet loss implementing the following architectures: Google/Inception architecture, 3D Convolutional Network (C3D), and a 2-D Long short-term memory (LSTM) Recurrent Neural Network. We make use of still images and sequences from videos for training the networks and compare the performances implementing the above architectures. The dataset used was the YouTube Face Database designed for investigating the problem of face recognition in videos. The contribution of this paper is two-fold: to begin, the experiments have established 3-D Convolutional networks and 2-D LSTMs with the contrastive loss on image sequences do not outperform Google/Inception architecture with contrastive loss in top $n$ rank face retrievals with still images. However, the 3-D Convolution networks and 2-D LSTM with triplet Loss outperform the Google/Inception with triplet loss in top $n$ rank face retrievals on the dataset; second, a Support Vector Machine (SVM) was used in conjunction with the CNNs' learned feature representations for facial identification. The results show that feature representation learned with triplet loss is significantly better for n-shot facial identification compared to contrastive loss. The most useful feature representations for facial identification are from the 2-D LSTM with triplet loss. The experiments show that learning spatio-temporal features from video sequences is beneficial for facial recognition in videos.