Joint Face Detection and Facial Motion Retargeting for Multiple Faces

TL;DR

This paper introduces an end-to-end neural network that simultaneously detects multiple faces and retargets facial motion, achieving high accuracy and robustness while improving speed over existing methods.

Contribution

It proposes a novel multitask learning framework for joint face detection and 3D facial motion retargeting in a single network, eliminating the need for separate detection and retargeting steps.

Findings

High face detection accuracy achieved

Robust to extreme expressions and poses

Faster than state-of-the-art methods

Abstract

Facial motion retargeting is an important problem in both computer graphics and vision, which involves capturing the performance of a human face and transferring it to another 3D character. Learning 3D morphable model (3DMM) parameters from 2D face images using convolutional neural networks is common in 2D face alignment, 3D face reconstruction etc. However, existing methods either require an additional face detection step before retargeting or use a cascade of separate networks to perform detection followed by retargeting in a sequence. In this paper, we present a single end-to-end network to jointly predict the bounding box locations and 3DMM parameters for multiple faces. First, we design a novel multitask learning framework that learns a disentangled representation of 3DMM parameters for a single face. Then, we leverage the trained single face model to generate ground truth 3DMM parameters for multiple faces to train another network that performs joint face detection and motion retargeting for images with multiple faces. Experimental results show that our joint detection and retargeting network has high face detection accuracy and is robust to extreme expressions and poses while being faster than state-of-the-art methods.