Towards Fast, Accurate and Stable 3D Dense Face Alignment

TL;DR

This paper introduces 3DDFA-V2, a fast, accurate, and stable 3D dense face alignment method that balances speed and precision, suitable for real-time applications on CPU.

Contribution

The paper presents a novel regression framework with a lightweight backbone, meta-joint optimization, and virtual synthesis for improved speed, accuracy, and stability.

Findings

Runs at over 50fps on a single CPU core

Outperforms state-of-the-art models in accuracy and stability

Validated on multiple challenging datasets

Abstract

Existing methods of 3D dense face alignment mainly concentrate on accuracy, thus limiting the scope of their practical applications. In this paper, we propose a novel regression framework named 3DDFA-V2 which makes a balance among speed, accuracy and stability. Firstly, on the basis of a lightweight backbone, we propose a meta-joint optimization strategy to dynamically regress a small set of 3DMM parameters, which greatly enhances speed and accuracy simultaneously. To further improve the stability on videos, we present a virtual synthesis method to transform one still image to a short-video which incorporates in-plane and out-of-plane face moving. On the premise of high accuracy and stability, 3DDFA-V2 runs at over 50fps on a single CPU core and outperforms other state-of-the-art heavy models simultaneously. Experiments on several challenging datasets validate the efficiency of our method. Pre-trained models and code are available at https://github.com/cleardusk/3DDFA_V2.