Input Aggregated Network for Face Video Representation

TL;DR

This paper introduces a novel input aggregated network architecture that learns fixed-length representations from variable-length face videos, improving face video recognition performance.

Contribution

The work proposes an end-to-end network with aggregation and mapping units to effectively represent face videos, a novel approach compared to traditional image-based recognition methods.

Findings

Effective face video representation demonstrated on public datasets

Outperforms existing methods in face video recognition tasks

End-to-end learning system successfully captures video-specific features

Abstract

Recently, deep neural network has shown promising performance in face image recognition. The inputs of most networks are face images, and there is hardly any work reported in literature on network with face videos as input. To sufficiently discover the useful information contained in face videos, we present a novel network architecture called input aggregated network which is able to learn fixed-length representations for variable-length face videos. To accomplish this goal, an aggregation unit is designed to model a face video with various frames as a point on a Riemannian manifold, and the mapping unit aims at mapping the point into high-dimensional space where face videos belonging to the same subject are close-by and others are distant. These two units together with the frame representation unit build an end-to-end learning system which can learn representations of face videos for the specific tasks. Experiments on two public face video datasets demonstrate the effectiveness of the proposed network.