Unsupervised Domain Adaptation for Face Recognition in Unlabeled Videos

TL;DR

This paper introduces an unsupervised domain adaptation framework that enhances video face recognition by transferring knowledge from labeled images, reducing domain gaps, and improving accuracy on challenging video datasets.

Contribution

It proposes a novel feature-level domain adaptation approach combining knowledge distillation, synthetic data augmentation, and adversarial learning for improved video face recognition.

Findings

Achieves state-of-the-art accuracy on YouTube Faces and IJB-A datasets.

Effectively suppresses video artifacts like pose, illumination, and occlusion.

Each module significantly contributes to the overall performance improvement.

Abstract

Despite rapid advances in face recognition, there remains a clear gap between the performance of still image-based face recognition and video-based face recognition, due to the vast difference in visual quality between the domains and the difficulty of curating diverse large-scale video datasets. This paper addresses both of those challenges, through an image to video feature-level domain adaptation approach, to learn discriminative video frame representations. The framework utilizes large-scale unlabeled video data to reduce the gap between different domains while transferring discriminative knowledge from large-scale labeled still images. Given a face recognition network that is pretrained in the image domain, the adaptation is achieved by (i) distilling knowledge from the network to a video adaptation network through feature matching, (ii) performing feature restoration through synthetic data augmentation and (iii) learning a domain-invariant feature through a domain adversarial discriminator. We further improve performance through a discriminator-guided feature fusion that boosts high-quality frames while eliminating those degraded by video domain-specific factors. Experiments on the YouTube Faces and IJB-A datasets demonstrate that each module contributes to our feature-level domain adaptation framework and substantially improves video face recognition performance to achieve state-of-the-art accuracy. We demonstrate qualitatively that the network learns to suppress diverse artifacts in videos such as pose, illumination or occlusion without being explicitly trained for them.