3D Facial Expression Reconstruction using Cascaded Regression

TL;DR

This paper introduces a cascaded regression approach for 3D facial expression reconstruction from a single image, effectively handling expression and pose variations while producing high-fidelity 3D face shapes.

Contribution

It presents a novel cascaded regression framework that estimates 3DMM parameters robustly from single images, improving over existing methods in handling pose and expression changes.

Findings

Robust to expression and pose variations

Reconstructs higher fidelity 3D face shapes

Outperforms existing methods in accuracy

Abstract

This paper proposes a novel model fitting algorithm for 3D facial expression reconstruction from a single image. Face expression reconstruction from a single image is a challenging task in computer vision. Most state-of-the-art methods fit the input image to a 3D Morphable Model (3DMM). These methods need to solve a stochastic problem and cannot deal with expression and pose variations. To solve this problem, we adopt a 3D face expression model and use a combined feature which is robust to scale, rotation and different lighting conditions. The proposed method applies a cascaded regression framework to estimate parameters for the 3DMM. 2D landmarks are detected and used to initialize the 3D shape and mapping matrices. In each iteration, residues between the current 3DMM parameters and the ground truth are estimated and then used to update the 3D shapes. The mapping matrices are also calculated based on the updated shapes and 2D landmarks. HOG features of the local patches and displacements between 3D landmark projections and 2D landmarks are exploited. Compared with existing methods, the proposed method is robust to expression and pose changes and can reconstruct higher fidelity 3D face shape.