Real-time 3D Facial Tracking via Cascaded Compositional Learning

TL;DR

This paper introduces a cascade of globally-optimized boosted ferns (GoMBF) for real-time 3D facial tracking from monocular RGB videos, achieving high performance with less training data and lower computational cost.

Contribution

It presents a novel cascade of deep compositional boosted ferns (GoMBF) that effectively regress multi-modal facial motion parameters in real-time.

Findings

Competitive tracking performance on in-the-wild videos

Requires less training data than state-of-the-art methods

Faster learning and inference compared to traditional boosted ferns

Abstract

We propose to learn a cascade of globally-optimized modular boosted ferns (GoMBF) to solve multi-modal facial motion regression for real-time 3D facial tracking from a monocular RGB camera. GoMBF is a deep composition of multiple regression models with each is a boosted ferns initially trained to predict partial motion parameters of the same modality, and then concatenated together via a global optimization step to form a singular strong boosted ferns that can effectively handle the whole regression target. It can explicitly cope with the modality variety in output variables, while manifesting increased fitting power and a faster learning speed comparing against the conventional boosted ferns. By further cascading a sequence of GoMBFs (GoMBF-Cascade) to regress facial motion parameters, we achieve competitive tracking performance on a variety of in-the-wild videos comparing to the state-of-the-art methods, which require much more training data or have higher computational complexity. It provides a robust and highly elegant solution to real-time 3D facial tracking using a small set of training data and hence makes it more practical in real-world applications.